Add dittosh server group — Ditto Server (portal) HTTP API client

.env.sample

@@ -1,3 +1,5 @@
 DATABASE_ID=
 OFFLINE_TOKEN=
-EXPIRE_ON=
+EXPIRE_ON=
+DITTOSH_SERVER_URL=https://xxxx.cloud.dittolive.app/<app-id>
+DITTOSH_SERVER_API_KEY=<api-key-from-portal>

AGENTS.md

@@ -6,16 +6,19 @@ The Ditto CLI: an npm/Homebrew-installable TypeScript CLI (binary `dittosh` —
 
 ## Hard rules (from the spec — do not regress)
 
-- **Offline-only.** Never call `startSync()`. No `--sync`, no user-supplied credentials (`--app-id`/`--license`, config commands). Every install shares one app ID — sync would leak data between users on a LAN.
+- **Offline-only local store.** Never call `startSync()`. No `--sync`, no user-supplied SDK credentials (`--app-id`/`--license`, config commands). Every install shares one app ID — sync would leak data between users on a LAN. (The `server` group below is the explicit exception in kind, not in mechanism: it's a plain HTTPS client for the portal's HTTP API with user-provided portal API keys — it never starts sync and never touches the local store or the SDK token.)
 - **Dev credentials:** repo-root `.env` (`DATABASE_ID`, `OFFLINE_TOKEN`, `EXPIRE_ON`; aliases `DQL_OFFLINE_LICENSE`/`DITTO_APP_ID`), gitignored, honored only in dev builds. Release builds (`RELEASE=true npm run build`) ignore env credentials and use the stamped, obfuscated embedded token (`scripts/stamp-token.ts`, M8).
+- **Server credentials (`dittosh server`):** `--url`/`--api-key` flags > shell env `DITTOSH_SERVER_URL`/`DITTOSH_SERVER_API_KEY` (aliases `DITTO_CLOUD_URL`/`DITTO_API_KEY`) > cwd `.env`. Missing → exit 3. The API key is never printed and is redacted from error messages.
 - **stdout is sacred.** Query results are the only thing on stdout (JSON when piped). Progress, warnings, banners, SDK logs → stderr. Never break this (jq composability is a feature).
 - One DQL statement per `store.execute` call; no trailing `;`.
-- Exit codes: `0` ok · `1` query/DQL error · `2` usage · `3` platform/token · `4` data-dir lock.
+- Exit codes: `0` ok · `1` query/DQL/API error · `2` usage · `3` platform/token/server-config/auth/connection · `4` data-dir lock.
 - Colors off when `NO_COLOR`, `CI`, `--no-color`, or non-TTY.
 
 ## Layout
 
-- `src/cli/` — commander entry (`index.ts`), injected version (`version.ts`, tsup `define`), `groups/` per command group (`dql`, later `skills`, `system`)
+- `src/cli/` — commander entry (`index.ts`), injected version (`version.ts`, tsup `define`), `groups/` per command group (`dql`, `server`, `skills`, `system`)
+- `src/cli/groups/server/` — `dittosh server` wiring: `common.ts` (flags/connect/error mapping/confirm), `store.ts` (execute/remote-execute), `attachments.ts`, `rbac.ts` (roles/users), `webhooks.ts`, `doctor.ts`. Thin glue; logic lives in `src/server/`. The legacy pre-DQL store API (find/findbyid/count/write) is deliberately NOT implemented — `server execute` covers it.
+- `src/server/` — portal HTTP API: `config.ts` (flags > shell env > cwd `.env`; URL normalize; sources), `client.ts` (`PortalClient`, injectable `FetchLike`, `PortalApiError`/`PortalConnectionError`, key redaction), `run.ts` (execute/remote-execute rendering through `src/render/`).
 - `src/config/` — data-dir resolution (`--data-dir` > `DITTOSH_DATA_DIR` > OS default), config dir (`DITTOSH_CONFIG_DIR` > OS default; env-paths caches homedir at module load, so tests must use this override, not `$HOME`), persisted state (one-time warnings, update cache)
 - `src/identity/` — token loading (dev env / release reassembly), expiry
 - `src/ditto/session.ts` — the only SDK touchpoint: init/open/close, log taming, lock mapping
@@ -47,9 +50,9 @@ scripts/install-release.sh # stamp token → RELEASE=true build → npm i -g . (
 
 ## Testing conventions
 
-- **unit** (`tests/unit`): no SDK. Fast; snapshot-friendly (`FORCE_COLOR=0` in setup).
+- **unit** (`tests/unit`): no SDK. Fast; snapshot-friendly (`FORCE_COLOR=0` in setup). Server-group tests inject a mock `FetchLike` via `registerServerGroup(cmd, { fetchImpl })` — no network; they scrub `DITTOSH_SERVER_*`/`DITTO_CLOUD_URL`/`DITTO_API_KEY` and `chdir` to an empty tmpdir because `tests/setup/env.ts` loads the repo `.env` (which may hold REAL portal credentials).
 - **integration** (`tests/integration`): real offline Ditto in a fresh tmpdir per file. Skip-gated on dev credentials via `tests/helpers/credentials.ts` (`hasDevCredentials`, `NO_CREDENTIALS`). `fileParallelism: false` — the native module holds process-wide state.
-- **e2e** (`tests/e2e`): execa spawning `node --import tsx --env-file=.env src/cli/index.ts`. Assert exit codes and both stdout/stderr separately. Each test uses its own tmp `-d` data dir.
+- **e2e** (`tests/e2e`): execa spawning `node --import tsx --env-file=.env src/cli/index.ts`. Assert exit codes and both stdout/stderr separately. Each test uses its own tmp `-d` data dir. `server.test.ts` runs a local `node:http` mock Ditto Server; spawns pass an explicit env with execa **`extendEnv: false`** (the v10 name — v9's `extend`) so real `.env` credentials never leak into a test run.
 - New user-facing command ⇒ e2e coverage. New logic ⇒ unit coverage. New SDK behavior ⇒ integration coverage.
 - **Coverage is a hard gate: ≥ 85%** statements/branches/functions/lines, enforced by `npm run coverage` (unit + integration projects, thresholds in `vitest.config.ts`) and in CI. `src/cli/index.ts` (process entry) is excluded deliberately — e2e covers it; v8 can't see subprocesses. Keep CLI glue thin: logic lives in injectable, unit-testable modules (see `doctor.ts`, `batch.ts`, `repl-core.ts`, `run.ts`).
 
@@ -59,3 +62,4 @@ scripts/install-release.sh # stamp token → RELEASE=true build → npm i -g . (
 - `DittoConfig(appId, { mode: "smallPeersOnly" }, dir)` + `Ditto.open` + `setOfflineOnlyLicenseToken` — verified (Spike A). EXPLAIN → first item `plan`; PROFILE → trailing `~request_profile` item.
 - The native tracing bootstrap writes ~7 WARN/INFO lines to **stderr** at `sdk.init()` (fd-level, not suppressible from JS). Cosmetic only; stdout is clean.
 - Two processes on one data dir → "File already locked" → mapped to `LockError` (exit 4).
+- Portal HTTP API (verified live on the retail app): Big Peer requires `FROM` in SELECT (`SELECT 1` → 400), so `server doctor` probes with `system:collections`; GET `/auth/roles` answers two wire shapes (bucketed + cursor-paged) — both normalized.

README.md

@@ -23,7 +23,7 @@ brew install getditto/tap/dittosh # Homebrew (macOS/Linux)
 
 The binary is `dittosh`. Requires Node.js ≥ 20 for npm installs. Supported platforms (matching the Ditto Node SDK): **macOS arm64, Linux x64/arm64, Windows x64**. Intel Macs (darwin-x64) are not supported by SDK 5.1.0.
 
-The CLI ships with a built-in offline license and runs entirely locally — no account, no credentials, no sync. `startSync()` is never called. All your data lives in one local directory (see [Data directory](#data-directory)).
+The CLI ships with a built-in offline license and runs entirely locally — no account, no credentials, no sync. `startSync()` is never called. All your data lives in one local directory (see [Data directory](#data-directory)). (The [`dittosh server`](#dittosh-server--ditto-server-over-http) group is the one exception in kind: a plain HTTPS client for your Ditto Cloud app's HTTP API with *your* portal API key — it never starts sync and never touches the local store.)
 
 ## Quickstart
 
@@ -134,6 +134,48 @@ dittosh dql dataset reset retail --yes            # evict the dataset's collecti
 
 `dataset run` prints the resolved statement (on stderr, so stdout stays clean), then executes it. Query names resolve across datasets; ambiguous names list the matches. `--setup` applies the entry's index DDL first; write-category catalog queries require `--yes` and clean up after themselves. `--seed <n>` reproduces a dataset exactly; changing seeds adds new documents (reset first for a clean slate).
 
+### `dittosh server` — Ditto Server over HTTP
+
+Run DQL against your Ditto Cloud app (the Big Peer behind it) over the portal HTTP API — the same API the portal's DQL editor uses. No local store involved; nothing syncs.
+
+```bash
+# configure once (or use flags every call)
+export DITTOSH_SERVER_URL=https://xxxx.cloud.dittolive.app/your-app-id
+export DITTOSH_SERVER_API_KEY=your-api-key
+
+# validate the setup before scripting
+dittosh server doctor
+
+# run DQL on the server
+dittosh server execute "SELECT * FROM customers LIMIT 5"
+dittosh server execute "SELECT * FROM orders WHERE total > :t" -p t=100
+dittosh server execute "INSERT INTO cars DOCUMENTS (:car)" --args '{"car":{"_id":"c1","make":"Toyota"}}'
+cat batch.sql | dittosh server execute          # one HTTP call per statement
+dittosh server execute "EXPLAIN SELECT * FROM orders WHERE store_id = 's1'"
+```
+
+Results render exactly like `dittosh dql` (table on TTY, JSON when piped, `-o`/`--format`/`--max-rows`/pager/`--time`). Mutations print `OK` and a `(transactionId … · N documents mutated)` note on stderr.
+
+**Configuration** is resolved in this order (first hit wins):
+
+1. **Flags:** `--url <url>` / `--api-key <key>`
+2. **Shell env:** `DITTOSH_SERVER_URL` / `DITTOSH_SERVER_API_KEY` (aliases from the Ditto docs also work: `DITTO_CLOUD_URL` / `DITTO_API_KEY`)
+3. **A `.env` file in the current directory** (never overrides the real environment)
+
+Find the URL in the portal → your app → "Connecting via HTTP" → **Cloud URL Endpoint** (looks like `xxxx.cloud.dittolive.app/<app-id>`). Create API keys in the portal → your app → **Auth → New API key**. Prefer env/`.env` over `--api-key` — argv is visible in `ps` and shell history. The URL must be `https://` (cleartext `http://` is rejected for non-local hosts; loopback is exempt for local testing).
+
+| Command | What it calls |
+|---|---|
+| `server execute [statement]` (alias `exec`) | `POST /api/v5/store/execute` — any DQL: SELECT/INSERT/UPDATE/DELETE/EXPLAIN/…; `-e`/`-f`/stdin batch, `-p`/`--args`, `--txn-id`, `--api-version v4\|v5`, `--timeout` (default 120s) |
+| `server remote-execute <statement>` | `POST /api/v5/sync/remote_execute` — run DQL on connected edge peers (statement must start with `SYNC CONTEXT`) |
+| `server attachment upload <file>` / `get <id>` | `POST /api/v4/attachments/upload` (multipart) / `GET /api/v4/attachments/{id}` (raw bytes; refuses to dump binary to a terminal — use `-o` or pipe) |
+| `server roles list/create/delete` | `/api/v4/auth/roles` — Big Peer RBAC roles (portal-internal API) |
+| `server users list/set-roles/delete` | `/api/v4/auth/users` — app users and their role sets (portal-internal API) |
+| `server webhook-secrets list/create/rotate/delete` | `/api/v4/auth/webhook/secret` — auth-webhook HMAC secrets (portal-internal API) |
+| `server doctor` | config → connection → auth probe; exit 3 on any failure |
+
+The legacy pre-DQL store API (`find`/`findbyid`/`count`/`write`) is deliberately not supported — `server execute` covers it with full DQL. Every command's `--help` documents its request body and examples (the RBAC/webhook endpoints aren't publicly documented — the help text is the reference). Batch mode: auth/connection failures stop the batch with exit 3 even under `--continue-on-error`; a timeout does *not* mean the statement failed — a mutation may still commit server-side.
+
 ### Global flags
 
 `--no-color`, `--quiet` (suppress informational notes), `--no-update-check` (planned; update flow lands in a later milestone).
@@ -180,9 +222,9 @@ dittosh dql --advise --apply -y "SELECT …"  # apply them
 | Code | Meaning |
 |---|---|
 | 0 | ok |
-| 1 | query/DQL error |
+| 1 | query/DQL error · server API error · server timeout (`server`) |
 | 2 | usage error (bad flags, missing file, ambiguous dataset query, …) |
-| 3 | platform/token/data-dir problem (unsupported OS/arch, expired license, unwritable dir) |
+| 3 | platform/token/data-dir problem (unsupported OS/arch, expired license, unwritable dir) · server config missing/invalid, auth rejected, or unreachable (`server`) |
 | 4 | data directory locked by another process |
 
 ## REPL