8. API Reference
CLI:
Local S3 commands use --root to identify the cluster root and discover its access endpoint:
crowdb-cli s3 cluster start --root <path>— create or restartcrowdb-cli s3 cluster status --root <path>— inspect process livenesscrowdb-cli s3 cluster stop --root <path>— stop and preserve datacrowdb-cli s3 cluster delete --root <path>— stop and delete permanentlycrowdb-cli s3 bucket put --root <path> <bucket>crowdb-cli s3 bucket delete --root <path> <bucket>crowdb-cli s3 bucket list --root <path>crowdb-cli s3 bucket get --root <path> <bucket>crowdb-cli s3 object put --root <path> <bucket> <key> [--file <path> | --text <content> | --random-size <bytes>]crowdb-cli s3 object get --root <path> <bucket> <key> [--output <file>] [--range <start-end>]crowdb-cli s3 object delete --root <path> <bucket> <key>crowdb-cli s3 object head --root <path> <bucket> <key>crowdb-cli s3 object list --root <path> <bucket> [--prefix <prefix>] [--limit <n>] [--continuation <token>]
Lower-level management commands accept --system-ip <addr> (default
127.0.0.1) and --system-port <port> (default 10000).
crowdb-cli cluster status— servers + store/group summarycrowdb-cli cluster topology— full logical + physical hierarchycrowdb-cli cluster inspect <id>—s<sid>,s<sid>/g<gid>,s<sid>/g<gid>/r<rid>, or<node-id>crowdb-cli cluster init --nodes n1,n2,...— initialize cluster (system group)crowdb-cli rack add --id <id> [--name <name>]crowdb-cli rack remove --id <id>crowdb-cli rack listcrowdb-cli node add --id <id> --rack <rack> [--host <host>] [--ssh-user <user>]crowdb-cli node remove --id <id>crowdb-cli node listcrowdb-cli node ping <node>crowdb-cli server deploy --node <id> --rest-port <p> --rpc-port <p>crowdb-cli server restart --node <id>crowdb-cli server stop --node <id>crowdb-cli server listcrowdb-cli store add --store-id <id> [--nodes n1,n2,...]crowdb-cli store remove --store-id <id>crowdb-cli store listcrowdb-cli store inspect --store-id <id>crowdb-cli paxos add --store-id <s> --group-id <g> --replica-id <r> --nodes n1,n2,...crowdb-cli paxos remove --store-id <s> --group-id <g>crowdb-cli paxos list --store-id <s>crowdb-cli paxos inspect --store-id <s> --group-id <g>crowdb-cli replica add --store-id <s> --group-id <g> --node <n> [--replica-id <r>]crowdb-cli replica remove --store-id <s> --group-id <g> --replica-id <r>crowdb-cli kv put --store-id <s> --group-id <g> --key <k> --value <v>crowdb-cli kv get --store-id <s> --group-id <g> --key <k>crowdb-cli kv delete --store-id <s> --group-id <g> --key <k>crowdb-cli kv scan --store-id <s> --group-id <g> --prefix <p> [--limit <n>]— list scan (fast, latest values, S3-list semantics)crowdb-cli snapshot create --store-id <s> --group-id <g>— pin a point-in-time snapshotcrowdb-cli snapshot list --store-id <s> --group-id <g>— list active snapshotscrowdb-cli snapshot scan --store-id <s> --group-id <g> --handle <h> --prefix <p> [--limit <n>] [--start-after <k>]— scan a pinned snapshotcrowdb-cli snapshot release --store-id <s> --group-id <g> --handle <h>— release a snapshot
curl:
S3 data plane
These endpoints use S3_ENDPOINT, whose local default is
http://127.0.0.1:16000. Cluster lifecycle and benchmark operations do not
currently have HTTP endpoints.
| Operation | Endpoint |
|---|---|
| List buckets | GET / |
| Create bucket | PUT /{bucket} |
| Inspect bucket | HEAD /{bucket} |
| Delete empty bucket | DELETE /{bucket} |
| Put or replace object | PUT /{bucket}/{key} |
| Get object | GET /{bucket}/{key} |
| Get inclusive range | GET /{bucket}/{key} with Range: bytes={start}-{end} |
| Inspect object | HEAD /{bucket}/{key} |
| Delete object | DELETE /{bucket}/{key} |
| List objects | GET /{bucket}?list-type=2&prefix=...&max-keys=... |
| Continue object list | GET /{bucket}?list-type=2&continuation-token=... |
Cluster lifecycle
| Operation | Endpoint |
|---|---|
| Initialize cluster | POST /api/cluster/init |
Physical topology
| Operation | Endpoint |
|---|---|
| List racks | GET /api/racks |
| Create rack | POST /api/racks |
| Delete rack | DELETE /api/racks/{rack_id} |
| List nodes | GET /api/nodes |
| Add node | POST /api/nodes |
| Get node | GET /api/nodes/{id} |
| Remove node | DELETE /api/nodes/{id} |
| Ping node | POST /api/nodes/{id}/ping |
| Get server info | GET /api/nodes/{id}/server |
| Deploy server | POST /api/nodes/{id}/server/deploy |
| Restart server | POST /api/nodes/{id}/server/restart |
| Stop server | POST /api/nodes/{id}/server/stop |
Logical topology (stores and groups)
| Operation | Endpoint |
|---|---|
| List stores | GET /api/stores |
| Create store | POST /api/stores |
| Get store | GET /api/stores/{sid} |
| Remove store | DELETE /api/stores/{sid} |
| List groups | GET /api/stores/{sid}/groups |
| Create group | POST /api/stores/{sid}/groups |
| Get group view | GET /api/stores/{sid}/groups/{gid} |
| Remove group | DELETE /api/stores/{sid}/groups/{gid} |
| List replicas | GET /api/stores/{sid}/groups/{gid}/replicas |
| Add replica | POST /api/stores/{sid}/groups/{gid}/replicas |
| Get replica | GET /api/stores/{sid}/groups/{gid}/replicas/{rid} |
| Remove replica | DELETE /api/stores/{sid}/groups/{gid}/replicas/{rid} |
| Resolve leader endpoint | GET /api/stores/{sid}/groups/{gid}/endpoint |
KV data plane
| Operation | Endpoint |
|---|---|
| Get | GET /api/stores/{sid}/groups/{gid}/kv/get?key=... |
| Put | POST /api/stores/{sid}/groups/{gid}/kv/put |
| Delete | POST /api/stores/{sid}/groups/{gid}/kv/delete |
| Scan (list mode) | GET /api/stores/{sid}/groups/{gid}/kv/scan?prefix=...&limit=N |
| Create snapshot | POST /api/stores/{sid}/groups/{gid}/snapshots |
| List snapshots | GET /api/stores/{sid}/groups/{gid}/snapshots |
| Snapshot scan | GET /api/stores/{sid}/groups/{gid}/snapshots/{handle}/scan?prefix=...&limit=N&start_after=... |
| Release snapshot | DELETE /api/stores/{sid}/groups/{gid}/snapshots/{handle} |
| Set GC watermark | POST /api/stores/{sid}/groups/{gid}/gc-watermark |
Server management (per-node, internal)
| Operation | Endpoint |
|---|---|
| System init (bootstrap group 0) | POST /system/init |
| Add store | POST /stores |
| Remove store | DELETE /stores/{sid} |
| Add group | POST /stores/{sid}/groups |
| Remove group | DELETE /stores/{sid}/groups/{gid} |
| Add remote replicas | POST /stores/{sid}/groups/{gid}/remotes |
| Step down leader | POST /stores/{sid}/groups/{gid}/step-down |
| Export topology | GET /topology |
| Health check | GET /health |
| Metrics | GET /metrics |
These endpoints are on the crowdb-kv-server management API (internal,
only called by crowdb-kv-client's KVClusterAdmin). The console's
POST /api/cluster/init orchestrates
/system/init across nodes and auto-finalizes.
9. Iceberg Catalog Foundation
The independent crowdb-iceberg binary exposes authenticated catalog configuration
only. Namespace, table and FileIO endpoints are not enabled. It uses an existing
healthy Group 0, Chunk-KV and chunk-storage deployment; S3 credentials and buckets
do not select or authorize an Iceberg catalog.
pixi run -- cargo build -p crowdb-access-server --bin crowdb-iceberg
export CROWDB_MANAGEMENT_SEEDS=127.0.0.1:10000
export CROWDB_ICEBERG_LISTEN=127.0.0.1:8181
Supply three distinct, randomly generated 32–256-character ASCII tokens through
your secret-management environment: CROWDB_ICEBERG_READ_TOKEN,
CROWDB_ICEBERG_MANAGE_TOKEN and CROWDB_ICEBERG_CLEAR_TOKEN. Configure every
instance consistently. Management credentials can rename/initialize; only the
clear credential can replace the catalog. All three can read configuration.
The listener is plain HTTP: keep it on a trusted loopback/private hop behind a
TLS-terminating proxy. Do not transmit bearer credentials over public plain HTTP.
Set CROWDB_ICEBERG_TOKEN to the appropriate management token for CLI commands.
Each mutation takes a fresh UUIDv7 request identity. Preserve both that identity
and the exact arguments when retrying an interrupted command.
export CROWDB_ICEBERG_TOKEN="$CROWDB_ICEBERG_MANAGE_TOKEN"
pixi run -- target/debug/crowdb-iceberg initialize "$INIT_UUIDV7" primary
pixi run -- target/debug/crowdb-iceberg status
pixi run -- target/debug/crowdb-iceberg rename "$RENAME_UUIDV7" renamed "$ACTIVE_EPOCH"
pixi run -- target/debug/crowdb-iceberg serve
Status reports CatalogId, activation epoch, name and phase. Rename preserves the CatalogId. The server validates dependencies and reconciles the root before opening its listener. Ctrl-C stops admission and drains accepted connections.
pixi run -- curl -H "Authorization: Bearer $CROWDB_ICEBERG_READ_TOKEN" \
http://127.0.0.1:8181/v1/config
Absent or empty warehouse selects the active catalog. A nonempty warehouse
returns 404 NoSuchWarehouseException. Unsupported endpoints return 406; all
table-format capabilities are false and HTTP idempotency is not advertised.
Clear makes the old domain inaccessible and selects a new empty catalog. It is
not physical erasure. Obtain the exact epoch and CatalogId from status, then
explicitly confirm both:
export CROWDB_ICEBERG_TOKEN="$CROWDB_ICEBERG_CLEAR_TOKEN"
pixi run -- target/debug/crowdb-iceberg clear "$CLEAR_UUIDV7" empty \
"$ACTIVE_EPOCH" "$ACTIVE_CATALOG_ID"
Admission returns 503 during maintenance. Default persisted limits require an 11-second grace after the durable fence is observed. Restart cannot shorten it. Another healthy instance resumes interrupted operations. An uncertain command must be retried with its original identity and input, not a newly generated key. Requests have a 24-hour retry window; expired identities are rejected. Bounded ledger-slot collisions can reject new operations without evicting live receipts.
Run backend restart, two-instance and official-client checks with
pixi run -e iceberg-e2e test-pyiceberg-e2e. This uses a separate disposable runtime
registry and leaves persistent local cluster reservations intact.