Development Setup
Prerequisites
- Go 1.26 or later (see
go.modfor the exact minimum) - Docker (for hermetic E2E tests using Testcontainers)
golangci-lint(for linting)- Node.js (for
markdownlint-cli2, run vianpxbyscripts/check.sh)
Clone the Repository
Build the Binary
bin/cloudstic.
Project Structure
Cloudstic CLI is organized into clear package boundaries:client.go(root) - PublicClientAPI for programmatic use. Re-exports types from internal packages via Go type aliases.cmd/cloudstic/- CLI entry point (package main).main.gois a thin dispatcher; each command lives in its owncmd_<name>.gofile, registered incommands.go’scommandRegistry(). See CLI Integration below.internal/engine/- Business logic for operations (backup, restore, prune, forget, diff, list, find). Each operation has a*Managerstruct.internal/core/- Repository-format types:Snapshot,Content,HAMTNode,RepoConfig, plusComputeJSONHash.FileMeta,SourceInfoandFileTypeare defined inpkg/sourceand aliased here, so the public Source contract does not depend on an internal package.internal/hamt/- Persistent Merkle Hash Array Mapped Trie backed by the object store.pkg/source/- TheSourceandIncrementalSourcecontract, plus shared helpers (ExcludeMatcher,TopoSortFolderChanges). Depends on nothing outside the standard library, so implementing a source pulls in no provider SDK. Implementations live in their own subpackages:pkg/source/{local,sftp,gdrive,onedrive}. See the Source Interface guide.internal/sourceoauth/- OAuth2 machinery shared by the Google Drive and OneDrive sources. Also holds the-Xldflags targets for the default OAuth client IDs; those symbol paths are mirrored in.goreleaser.ymland must move together, since the linker silently ignores-Xfor a symbol that does not exist.pkg/store/- TheObjectStorecontract, its capability interfaces (RangeGetter,ConcurrencyHinter,Unwrapper) and the order-independent wrappersQuotaStore/DebugStore. Depends on nothing outside the standard library, so implementing a custom backend pulls in no vendor SDK. Backends live inpkg/store/{local,s3,b2,sftp}/(local.New,s3.New, …); onlypkg/store/s3carries the AWS SDK.pkg/store/storetestholds shared test doubles.internal/storelayer/- The repository-format decorator chain (compression, encryption, metering, packfiles, key cache). Internal because its composition order is a security invariant — see the Store Decorator Stack below.pkg/crypto/- AES-256-GCM encryption, HKDF key derivation, BIP39 mnemonic recovery keys.pkg/keychain/- OS keychain integration and encryption key-slot helpers.internal/app/- Orchestration layer shared by the CLI and TUI (profile listing, health checks, backup actions).internal/tui/- The interactive terminal dashboard (Bubble Tea).pkg/secretref/- Thescheme://pathsecret-reference contract:Ref,Parse,Backend,Resolver,NewResolver,Error. Public so a third party can register a custom backend (Vault, a cloud KMS) from another module.pkg/secretref/backends/- The built-in backends (env://,file://,config-token://,keychain://,secret-service://,wincred://) plusDefault(), which returns a fresh map so callers extend the built-in set rather than replacing it.pkg/profile/- The backup-profiles YAML format (Config,Profile,Store,Auth,Load,Save). Separate from the client so profiles can be read and written without opening a repository.internal/workstation/- Workstation onboarding (Plan,Apply,Setup) and local source discovery. Internal: it is a CLI wizard, not a library capability.internal/paths/- Config-directory and token-path resolution.internal/pathmatch/- Glob matching for slash-separated paths, including**.internal/logger/,internal/retry/,internal/sftp/- Structured logging, retry/backoff helpers, and the shared SFTP client used by both the SFTP source and store.internal/ui/- Non-interactive console progress reporting and terminal helpers.
Build & Test Commands
Run All Tests
Docker is required for hermetic E2E tests. Tests will be skipped if
/var/run/docker.sock is not available.Run a Single Test
Run with Race Detector
-race during development.
Run the Full Check Script
go fmt- Format checkgolangci-lint run- Lintingmarkdownlint-cli2(vianpx) - Markdown lint, including this guide and every other.mdfile in the repogo test -race -count=1 ./...- Tests with race detection- Coverage report generation
Format Code
Lint Code
E2E Test Modes
E2E tests ine2e/ are controlled by the CLOUDSTIC_E2E_MODE environment variable:
hermetic(default) - Local filesystem + Testcontainers (MinIO, SFTP). Requires Docker.live- Real cloud vendor APIs (requires secrets in environment variables).all- Runs both hermetic and live tests.
Running Hermetic Tests
Running Live Tests
Live tests require cloud provider credentials (AWS, Backblaze B2, Google Drive, OneDrive, SFTP servers) configured via environment variables.
Running All Tests
Debugging
Enable Debug Logging
Append the-debug flag to any CLI command to enable verbose internal logging:
- Detailed timings for every
GET,PUT,LIST, andDELETEoperation - Cache hits/misses
- Memory management decisions
- Engine operation traces
Attach a Debugger
You can usedlv (Delve) to debug the CLI:
Profiling
Cloudstic supports standard Go profiling via hidden flags on any command:CPU Profiling
cpu.prof.goroutine- Goroutine dumpcpu.prof.block- Block profilecpu.prof.mutex- Mutex profile
Memory Profiling
View Profiles
Usego tool pprof to analyze profiles:
Development Best Practices
When Adding New Features
Always consider the following:1. Documentation
Check if user-facing documentation needs updates:docs/user-guide.md- Add command documentation with usage examples, flags, and descriptions.README.md- Update if the feature changes the quick start or high-level overview.- Code comments - Document public APIs, especially in
client.goand package interfaces.
2. Unit Tests
Add test coverage when it makes sense:- Always add tests for new public API methods (e.g.,
Client.*()methods). - Test both success and error cases.
- Test integration with encryption/compression if applicable.
- Use existing test patterns (see
client_test.go,internal/engine/*_test.go). - Mock stores are available in
internal/engine/mock_test.gofor testing.
3. Client API
For new operations, expose them via theClient struct:
- CLI commands should use
Clientmethods, not directly access stores. - This allows library users to programmatically use the functionality.
- Follow the pattern: define types/options, add a
Client.*()method, implement ininternal/engine/if complex.
4. CLI Integration
For new commands, don’t hand-write a flag parser or editmain.go. Instead:
- Add a
cmd_<name>.gofile next to the code it implements, following the pattern every existing command uses (cmd_backup.go,cmd_check.go,cmd_breaklock.go, …):declare<Name>Args(g *globalFlags) (*<name>Args, commandInput)declares the command’s complete input surface — flags viastringFlag/boolFlag/intFlag/valueFlag, positionals viarequiredPositional/optionalPositional. This also carries environment, secret, help, and completion metadata.run<Name>(r *runner, ctx context.Context, a *<name>Args) intis the command body. It writes output viar.out/r.errOut(neverfmt.Print) so it’s capturable in tests.- A
<name>Command() commandfunction declares the runnable leaf:leaf(name, summary, groups, declare<Name>Args, run<Name>). PassrepoCommandGroups(orbackupCommandGroupsfor a command that reads a source) to opt into only the global flags the command needs.
- Add the new
<name>Command()to the ordered list incommandRegistry()(cmd/cloudstic/commands.go) — that’s the single source of truthrunCmd()dispatches from,printUsage()rendersCOMMANDSfrom, and shell completion is generated from. Never editmain.go,usage.go, orcompletion.godirectly for a new command. - Mark any flag carrying a credential with
asSecret()so it never leaks into-houtput (TestSecretEnvValuesNeverAppearInHelpenforces this). - If the command changes root or per-command help text, regenerate the golden files:
go test ./cmd/cloudstic -run 'TestRootUsageGolden|TestCommandHelpGolden' -update.
5. Error Handling
Return descriptive errors:- Wrap errors with context using
fmt.Errorf("context: %w", err). - Provide actionable error messages to users.
- Distinguish between user errors and system errors.
- If a caller (CLI or library) needs to programmatically detect a specific failure mode — not just display it — define a sentinel error (
var ErrX = errors.New(...)) and wrap it with%w, rather than making the caller match on error text. Seeengine.ErrRepoLocked/cloudstic.ErrRepoLockedfor a recent example wired through to a CLI hint.
Example: Adding a New Command
Let’s say you want to add astats command that shows repository statistics.
Step 1: Add Client Method
run<Name> takes the shared runner (so output is capturable and r.client is injectable in tests) instead of building its own client and flag set; presentation lives in a separate print* function taking io.Writer first, never given access to runner.
Step 4: Register the Command
runCmd() in main.go), the cloudstic help listing, and shell completion all derive from this list.
Step 5: Add Tests
docs/user-guide.md and (if stats has notable flags) run go test ./cmd/cloudstic -run TestCommandHelpGolden -update to refresh its golden -h output.
Testing Guidelines
Choose the smallest test style that covers the behavior — the four styles below cover different layers, and picking the wrong one for a given change is a common review comment:- Unit tests for argument parsing, orchestration branches, domain logic, and individual error cases. Inject a
stubClient(cmd/cloudstic/stub_client_test.go) to test CLI command flow without a real repository. - Golden-file tests for deterministic
print*/render*presentation output, where the exact full text is the contract. - Testscript tests for whole-command behavior that crosses the process boundary: flag ordering, stdout/stderr separation, exit codes, filesystem effects.
- E2E tests (
e2e/, Testcontainers-backed) for behavior that needs a real backend (MinIO, SFTP) end to end.
Test Coverage
Aim for high test coverage, especially for:- Public API methods in
client.go - Engine logic in
internal/engine/ - Store implementations in
pkg/store/ - Crypto operations in
pkg/crypto/
Test Patterns
Unit Tests
Use mock stores for isolated engine testing:stubClient instead of a real repository — see TestRunStats in the command example above.
Golden-File Tests
Write deterministic output to a buffer and compare it againstcmd/cloudstic/testdata/*.golden with assertGolden:
-update, and review the diff as part of the change:
Testscript Tests
Add a hermetic.txtar script under cmd/cloudstic/testdata/scripts/ for whole-process behavior. Prefer local stores/sources so the script needs no network access, credentials, or Docker:
Integration Tests
Use real stores with temporary directories:E2E Tests
Use Testcontainers for hermetic E2E tests:Before Committing
Always run the full check script:- Code is formatted correctly
- No linting errors
- All tests pass
- Race conditions are detected
- Coverage is adequate
Architecture Overview
Store Layering
Stores are composed as a decorator chain (from outermost to innermost):- CompressedStore - zstd compression on write, auto-detects zstd/gzip/raw on read.
- EncryptedStore - AES-256-GCM. Passes through objects under
keys/prefix unencrypted. - MeteredStore - Tracks bytes written for reporting.
- PackStore (optional) - Bundles small objects (less than 512KB) into 8MB packfiles to reduce API calls. Each packfile ends with a self-describing footer, so its catalog (
index/packs) is a rebuildable cache rather than the sole source of truth — see RFC 0018 if you’re touching this layer. - KeyCacheStore - Caches key existence in a temporary bbolt database.
- Backend -
local.Store,s3.Store,b2.Store, orsftp.Store, each in its own subpackage underpkg/store/.
Repository Compatibility
- Backward compatibility is permanent. A repository written by any released version must stay readable by every later version (
list,ls,check,cat,diff,restore). There is no deprecation window for reads. - Forward compatibility isn’t guaranteed, but failure must be safe. An older build may be unable to read a newer repository, but it must never misread it as empty or valid — “cannot decode” must always surface as an error, never as “no entries,” because that’s exactly the condition that lets a garbage collector delete a live repository.
- The version gate (
core.RepoFormatVersion/core.MaxSupportedRepoFormatininternal/core/models.go) gates every repository open. Raise it only when a change would make a repository unreadable or misreadable by earlier builds. - Changing the on-disk format requires: keeping older layouts readable, upgrading only opportunistically (never requiring a migration to read), committing a fixture from the last release in the old format, deciding on the version gate, adding the baseline to
docs/compatibility.md’s table, and stating in the PR what older builds do when they meet the new format — verified by running an old binary, not by reasoning about it.e2e/feature_legacy_repo_test.goenforces the fixture and doc-table requirements.
Backup Flow
BackupManageracquires a shared lock, loads the previous snapshot (if any) for its source identity.- Source is scanned via
Walk()(full) orWalkChanges()(incremental). - New/changed files are chunked using FastCDC, content-addressed, and uploaded.
- The HAMT tree is updated with new filemeta refs.
TransactionalStorebuffers all intermediate HAMT nodes and only flushes reachable ones from the final root. - A new
Snapshotobject is written, andindex/latestis updated.
Encryption Model
- On
init, a random 32-byte master key is generated and wrapped into key slots (password-based via scrypt, platform key, KMS-wrapped platform key, or BIP39 recovery key). - Key slots are stored under
keys/prefix, which theEncryptedStorepasses through unencrypted. - An HMAC dedup key is derived from the encryption key via HKDF for content-addressing without exposing plaintext hashes.
Naming Conventions
Commits & PR titles use a Conventional Commit prefix —type: imperative summary, or type(scope): …. Lowercase the summary after the colon, no trailing period, keep it short (~72 chars) and specific about what changed. PRs are squash-merged, so the PR title becomes the commit subject — give both the same form.
- Types:
feat,fix,refactor,test,docs,chore,perf,ci. - An optional scope names the area:
feat(tui):,fix(completion):.
<type>/<kebab-slug> (e.g. feat/tui-profile-history), matching the commit type.
Issue titles carry no conventional-commit prefix — the type lives in the label. Lead with an imperative verb (Add …, Convert …) or an Area: scanning prefix, no trailing period.
RFCs: a substantial feature gets a design record under rfcs/NNNN-kebab-slug.md (zero-padded, next free number). The proposal PR/commit is rfc: <summary> (RFC NNNN); the implementation PR/commit is a standard type: prefix with a trailing (RFC NNNN) reference, e.g. feat: unified source identity (RFC 0009). See rfcs/README.md for the index and RFC 0010 as a template.
Pull Request Guidelines
Before Submitting
- Run tests:
./scripts/check.sh - Update documentation: Add/update user guide, README, and code comments
- Add tests: Cover new functionality with unit/golden-file/testscript tests as appropriate
- Format code:
go fmt ./... - Lint code:
golangci-lint run ./...
PR Description
Two sections:Summary bullets high-signal — what changed and why, not a file-by-file diff. Under Verification, paste the exact commands you ran, e.g.:
Creating an Issue
Issues use four sections —## Context (current state, with backtick paths to concrete files/functions), ## Goal (desired end state, one or two sentences), ## Scope (bullet list of concrete changes), and ## Acceptance Criteria (verifiable outcomes, always ending with the exact test/lint commands that must pass). Apply exactly one type label (bug, enhancement, refactor, tech debt, chore, test, documentation, rfc, tracking) plus one or more area/* labels.
Getting Help
If you have questions or need help:- Check
AGENTS.mdfor architecture details - Review existing code for patterns
- Open a GitHub issue for discussion
- Join our community chat (if available)