diff --git a/docs/proposals/db-tunnel.md b/docs/proposals/db-tunnel.md new file mode 100644 index 00000000..590fc7be --- /dev/null +++ b/docs/proposals/db-tunnel.md @@ -0,0 +1,292 @@ +# Proposal: `gddy db tunnel` — MySQL access over a WebSocket bridge + +Status: draft; implemented behind an experimental feature flag, seeking review +alongside the server-side changes it depends on. + +## Motivation + +Developers and support staff sometimes need to point an ordinary MySQL client +(`mysql`, DBeaver, an app's ORM) at the database behind a GoDaddy-hosted +application — to inspect data, run a migration, or debug an issue. There is no +first-class way to do that from `gddy` today. + +`gddy db tunnel` opens a local TCP port and, for each client connection, bridges +raw MySQL bytes over a single WebSocket to the application's per-app **agent**, +which dials the app's own database. The tunnel is a byte pump: it never parses +the MySQL protocol and never injects or inspects credentials, so MySQL +authentication and TLS are negotiated **end-to-end** between the client and the +real database, exactly as they would be for a direct connection. + +## How it works + +There are two planes: + +- **Control plane** — a single, out-of-band HTTPS call the CLI makes to the + GoDaddy hosting API to obtain a short-lived, app-scoped token and the URL of + the app's agent. This happens once, when the command starts. +- **Data plane** — the byte relay. For each local TCP connection the CLI opens + one WebSocket to the agent and pumps raw MySQL bytes in both directions. No + database traffic ever touches the hosting API. + +```mermaid +flowchart LR + client["MySQL client\n(mysql, DBeaver, app)"] + subgraph local["Developer machine"] + cli["gddy db tunnel\n(local TCP listener)"] + end + api["GoDaddy hosting API\n(token mint)"] + agent["App agent\n(WebSocket endpoint)"] + db[("App MySQL\nhost:port")] + + cli -.->|"① mint once: OAuth Bearer → {agent URL, token}"| api + client -->|"TCP :3306"| cli + cli -->|"② one WSS per TCP conn\nbinary frames = raw MySQL"| agent + agent -->|"connect(host, port)\nno credential injection"| db +``` + +Key properties, all enforced in code: + +- **One WebSocket per TCP connection.** No multiplexing, no session resumption. + When either end closes, the other is closed. +- **Binary frames carry raw MySQL bytes**, verbatim in both directions. Neither + side understands the MySQL wire protocol. +- **Host and port only.** The agent resolves the database's network location + from the app's own configuration and dials it. It never places a username, + password, or connection string into the byte stream — the client completes + the MySQL handshake, including auth and TLS, directly against the database. +- **The mint and the relay use different paths.** The one-time token mint is an + ordinary HTTPS API call; the MySQL bytes flow only over the WebSocket to the + agent. No MySQL bytes ever transit the hosting API. + +## Command surface + +`gddy db tunnel` is a streaming command +(`RuntimeCommandSpec::new_typed_streaming`) at tier `Mutate`. Flags: + +| Flag | Required | Default | Purpose | +| --- | --- | --- | --- | +| `--app-id ` | yes | — | The application/site to tunnel to. The CLI mints a token for this app and connects to its assigned agent. | +| `--port ` | no | `3306` | Local TCP port MySQL clients connect to. | +| `--listen-host ` | no | `127.0.0.1` | Local interface to bind. | + +The command authenticates with the CLI's own GoDaddy OAuth credential, stepped +up to the same scope that publishing a hosting deployment requires. It first +asks the hosting API for the app's assigned agent URL and a short-lived token, +then opens the tunnel to that URL, presenting the token as +`Authorization: Bearer`. Neither the agent URL nor the token is a user-supplied +flag — both come from the service, so there is nothing to paste and no endpoint +to wire up by hand. + +```console +$ gddy db tunnel --app-id +# then, in another shell: +$ mysql -h 127.0.0.1 -P 3306 -u -p +``` + +### Input validation and URL derivation + +`--app-id` is validated by `validate_app_id` at the very top of the command, +**before** the mint call — it is interpolated into both the token-request path +and the WebSocket route, so anything that could alter URL structure (`/`, `#`, +`?`, `%`, whitespace) is rejected up front rather than silently misrouting the +request to a confusing `404`/`405`. App ids are short slugs, so the allowlist is +ASCII letters, digits, `-` and `_`. + +`build_tunnel_ws_url(agent_url, app_id)` then turns the agent URL returned by the +mint call into the WebSocket URL for the tunnel: + +- `http` → `ws`, `https` → `wss`, `ws`/`wss` preserved; any other scheme is + rejected. +- Only scheme + host:port are taken from the returned URL; any path is discarded + and replaced with the fixed tunnel route. +- It re-runs `validate_app_id` (defense in depth). + +Validation and URL derivation both happen before a port is bound, so a bad +`--app-id` or a malformed agent URL fails fast — with a single terminal error +line — before any client is accepted. + +### Relay + +Per accepted connection, `handle_connection` sets `TCP_NODELAY`, opens one agent +WebSocket (`connect_agent`, attaching the minted token as +`Authorization: Bearer`), and calls `relay`. `relay` runs **each direction as +its own future**, joined by a final `select!` that returns as soon as either +side ends: + +- **Upstream** (client → agent) owns the WebSocket sink: it reads up to 64 KiB + from the TCP socket and sends one binary message, and on client EOF sends a + close frame (`client-closed`). Because it owns the sink, it also runs the + **10-second flush tick** that pushes out auto-queued pong replies while the + link is idle (the agent pings roughly every 30s and drops the tunnel on a + missed pong). +- **Downstream** (agent → client) owns the TCP write half: it writes binary + payloads, treats close/EOF as an agent-side close, ignores non-binary frames, + and maps a stream error to an error close. + +Running the two directions as separate futures is a deliberate correctness +choice: a single `select!` loop that awaited `send`/`write_all` **inline** would, +under simultaneous bidirectional backpressure, suspend the whole loop on one +direction's blocked write and stop draining the other — deadlocking the +connection and starving the keepalive flush. Byte counters are shared +`AtomicU64`s incremented **only after** a successful forward, so the reported +`bytesUp`/`bytesDown` never include a chunk that failed to reach the far side. +`relay` returns `(bytes_up, bytes_down, reason)`. + +The 64 KiB upstream chunk keeps outbound frames small; MySQL reassembles its own +packets from the byte stream regardless of framing. In the other direction, +`connect_agent` raises tungstenite's inbound frame limit from its 16 MiB default +to the agent's **32 MiB** cap (`connect_async_with_config`), so a large +downstream result frame is not rejected mid-query. + +### Events + +Progress is streamed as JSON events: an `authorize` step (`started`/`completed`) +around the token mint, `listening` (bound address, agent URL, app id), a `hint` +with the `mysql -h …` line, `connection` events (`open`/`close`/`error`, with +per-connection byte counts and a close reason), a `warning` on a failed +`accept`, a `shutdown` step on Ctrl-C, and a terminal `result` carrying the app +id and total accepted connections. Terminal errors reuse +`cli_engine::build_error_envelope` via `tunnel_error_event`, so `code`, +`message`, and `fix` match what a non-streaming command would render. +`map_ws_err` turns handshake failures into status-aware fixes +(401/403/404/429/503). + +### Feature gating + +The `db` module is registered with +`.with_feature_flag("db", Stage::Experimental)`, so it is **hidden at the global +`Stage::Ga` default** and revealed only in an environment whose resolved +`min_stage` is `experimental`. A test in `main.rs` +(`db_tunnel_is_gated_and_exposes_its_flags_when_revealed`) guards both halves: +hidden at GA, and — once revealed — `db tunnel --help` lists every flag. + +### Dependencies + +`tokio-tungstenite` (rustls, webpki roots — no native-tls) for the WebSocket +client, and `futures-util` (`std`, `sink`) for the split sink/stream. `bytes` +and `url` were already present. + +## Authentication and authorization + +The CLI never handles a raw agent token as user input. It authenticates the +*mint call* with its own GoDaddy OAuth credential; the hosting platform performs +the identity exchange server-side and hands back a short-lived, app-scoped token +that the CLI simply presents to the agent. + +Authorization is checked in depth: + +1. **OAuth scope at the mint endpoint.** The mint call requires the same OAuth + scope that publishing a hosting deployment does, and is rate-limited. An + unauthenticated or under-scoped caller never reaches the mint logic. +2. **App ownership at the hosting API.** The app is resolved scoped to the + authenticated customer; an app the caller does not own is not found, and the + request fails **before** any token is minted. +3. **App ownership again at the agent.** On the WebSocket upgrade the agent + re-validates the token and requires that the app the token is scoped to + matches the app in the connection path, so a valid token for one app cannot + open a tunnel to another. + +Beneath all of that, **MySQL's own authentication and TLS run end-to-end** and +are never short-circuited: the tunnel carries opaque bytes, so +`require_secure_transport=ON` and normal user/password/`GRANT` checks apply +exactly as they would for a direct connection. + +### The CLI holds only its OAuth credential + +The CLI holds only its GoDaddy **OAuth2 access token**. It never mints, parses, +stores, or pastes the app-scoped token the tunnel uses; it receives that token +from the hosting API at mint time and holds it in memory only for the lifetime +of the tunnel. Because the identity exchange and token signing are entirely +server-side, changing how the token is minted — including tightening its scope +later — needs **no CLI change**. + +## Security model + +- **No credential injection anywhere in the path.** The agent knows the DB host + and port and dials them; it never writes credentials into the relayed stream. + The MySQL client authenticates directly against the database, so end-to-end + MySQL auth and TLS (including `require_secure_transport=ON`) are preserved. +- **The destination is pinned server-side.** The client supplies no target host + — only `--app-id`. The agent derives the DB target from the app's own + configuration, so there is no SSRF surface. +- **Ownership is enforced in depth**, not just at the edge: an OAuth scope gate + and a customer-scoped app lookup before a token is minted, and a + token-vs-path app check at the agent before it dials. +- **The token is minted per-run and never persisted.** The CLI requests a + short-lived token when the command starts and holds it only in memory for the + lifetime of the tunnel; there is nothing to paste, store, or rotate by hand. +- **Blast-radius limits.** The mint endpoint is rate-limited; the agent bounds + idle and total connection duration, caps concurrent tunnels per app and in + total, and enforces a bounded per-frame size. +- **Logging hygiene.** Both the CLI and the agent emit connection-level metrics + (byte counts, duration, host/port, close reason) and never log payload bytes + or credentials. + +## Design decisions and rationale + +- **WebSocket to the app's agent, not a new proxy.** The agent already + terminates authenticated HTTP requests and sits inside the app-runtime network + with a route to the database. Riding the HTTP `upgrade` path reuses that edge + and those auth primitives without exposing a new public port. +- **Relay at the agent.** The agent is the only component that simultaneously + knows the app's DB host:port, has a network route to it, and already enforces + app ownership. Relaying anywhere else would duplicate DB-config resolution and + ownership checks. +- **Raw byte relay, no MySQL parsing.** Preserving end-to-end MySQL auth and TLS + keeps the DB credential story unchanged, avoids a MySQL-protocol parser as + attack surface, and keeps the tunnel protocol-agnostic. +- **Destination pinned by `--app-id`.** The client cannot ask the agent to dial + an arbitrary host, which removes the SSRF surface. +- **One WebSocket per TCP connection.** MySQL connections are independent; tying + each to its own WS keeps lifecycle trivial (close one, close the other) and + avoids multiplexing/head-of-line complexity. +- **Server-side mint that also returns the agent URL.** Keeps the app-scoped + token's signing authority server-side, reuses the platform's existing mint + primitive, and gives the CLI a one-command UX with nothing to paste. +- **Reuse the deployment scope.** Tunneling to an app's DB is a comparable level + of access to publishing a deployment, so no new OAuth scope needs to be + provisioned. +- **Ships gated off.** The CLI command is hidden at the GA default and revealed + only in experimental environments, and the server side is dark-launched, so + the feature can land and be validated without exposure. + +## Cross-team dependencies + +`gddy db tunnel` is the client half of a change that also needs two server-side +pieces, each tracked in its own repository and PR: + +- **A hosting-API endpoint** that, given an app the caller owns, mints a + short-lived, app-scoped token and returns the app's agent URL in one call. +- **An agent-side WebSocket handler** that authenticates that token, re-checks + app ownership, dials the app's database, and relays raw bytes. + +Both ship gated off by default, so the CLI command can be revealed independently +once the server pieces are enabled in a given environment. + +## Deferred work + +- **Token lifetime.** The token is minted once when the command starts and + reused for every WebSocket for the tunnel's lifetime (the mint response's + expiry is not yet consumed). A tunnel left open past the token's TTL keeps + serving its existing connections, but a *new* MySQL connection opened after + expiry fails the agent handshake with `401` and the command must be restarted. + Pre-emptive re-mint on expiry is a deliberate follow-up. +- **Full-path verification.** The CLI relay and URL/error mapping are + unit-tested, and the tunnel has been exercised end-to-end against the agent + handler in isolation. A run through the deployed mint endpoint and the + production agent edge is the remaining validation step. + +## Testing + +- **CLI unit tests** (`rust/src/db/tunnel.rs`): `build_tunnel_ws_url` for + http→ws / https→wss, scheme preservation, path replacement, and rejection of + bad schemes, missing hosts, and bad app ids — including a case that rejects + URL-metacharacter app ids (`#`, `?`, `%`, whitespace, `.`, `:`) before they + reach any request path; plus `tunnel_error_event` carrying + `code`/`message`/`fix`. +- **CLI client test** (`rust/src/hosting/nodejs/client.rs`): the mint call + (`get_agent_token`) POSTs with bearer auth and parses `{ agentUrl, token }` + from the response. +- **CLI gating test** (`rust/src/main.rs`): `db` hidden at the GA default and, + once revealed, `db tunnel --help` exposing its `--app-id`, `--port`, and + `--listen-host` flags. diff --git a/rust/Cargo.lock b/rust/Cargo.lock index cba5bb02..b2fdd2b8 100644 --- a/rust/Cargo.lock +++ b/rust/Cargo.lock @@ -365,6 +365,15 @@ dependencies = [ "generic-array", ] +[[package]] +name = "block-buffer" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa" +dependencies = [ + "hybrid-array", +] + [[package]] name = "block-padding" version = "0.3.3" @@ -489,7 +498,7 @@ version = "0.4.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "773f3b9af64447d2ce9850330c473515014aa235e6a783b02db81ff39e4a3dad" dependencies = [ - "crypto-common", + "crypto-common 0.1.7", "inout", ] @@ -644,6 +653,12 @@ dependencies = [ "crossbeam-utils", ] +[[package]] +name = "const-oid" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c" + [[package]] name = "convert_case" version = "0.10.0" @@ -846,6 +861,21 @@ dependencies = [ "typenum", ] +[[package]] +name = "crypto-common" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453" +dependencies = [ + "hybrid-array", +] + +[[package]] +name = "data-encoding" +version = "2.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06" + [[package]] name = "dbus" version = "0.9.12" @@ -909,11 +939,22 @@ version = "0.10.7" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ - "block-buffer", - "crypto-common", + "block-buffer 0.10.4", + "crypto-common 0.1.7", "subtle", ] +[[package]] +name = "digest" +version = "0.11.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" +dependencies = [ + "block-buffer 0.12.1", + "const-oid", + "crypto-common 0.2.2", +] + [[package]] name = "dirs" version = "6.0.0" @@ -1352,6 +1393,7 @@ dependencies = [ "domains-client", "fancy-regex", "flate2", + "futures-util", "globset", "httpmock", "iso_currency", @@ -1374,6 +1416,7 @@ dependencies = [ "tempfile", "thiserror", "tokio", + "tokio-tungstenite", "toml", "tracing", "tracing-subscriber", @@ -1442,7 +1485,7 @@ dependencies = [ "http", "httpdate", "mime", - "sha1", + "sha1 0.10.7", ] [[package]] @@ -1501,7 +1544,7 @@ version = "0.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "6c49c37c09c17a53d937dfbb742eb3a961d65a994e6bcdcf37e7399d0cc8ab5e" dependencies = [ - "digest", + "digest 0.10.7", ] [[package]] @@ -1583,6 +1626,15 @@ dependencies = [ "url", ] +[[package]] +name = "hybrid-array" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "707114b52a152fa7bdb290cd7cd5912d9467273b6d74e21b8d81aca1f8533f6b" +dependencies = [ + "typenum", +] + [[package]] name = "hyper" version = "1.11.0" @@ -1618,7 +1670,7 @@ dependencies = [ "tokio", "tokio-rustls", "tower-service", - "webpki-roots", + "webpki-roots 1.0.9", ] [[package]] @@ -3135,7 +3187,7 @@ dependencies = [ "wasm-bindgen", "wasm-bindgen-futures", "web-sys", - "webpki-roots", + "webpki-roots 1.0.9", ] [[package]] @@ -3616,7 +3668,18 @@ checksum = "a978451301f4db1d02937a4ab3ccce137717b81826e79b7d49ffe3244a13c3b8" dependencies = [ "cfg-if", "cpufeatures 0.2.17", - "digest", + "digest 0.10.7", +] + +[[package]] +name = "sha1" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aacc4cc499359472b4abe1bf11d0b12e688af9a805fa5e3016f9a386dc2d0214" +dependencies = [ + "cfg-if", + "cpufeatures 0.3.0", + "digest 0.11.3", ] [[package]] @@ -3627,7 +3690,7 @@ checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283" dependencies = [ "cfg-if", "cpufeatures 0.2.17", - "digest", + "digest 0.10.7", ] [[package]] @@ -4008,6 +4071,22 @@ dependencies = [ "tokio", ] +[[package]] +name = "tokio-tungstenite" +version = "0.30.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "17a073bfed563fa236697a068031408a93cd9522e08abf9933ead3e73411bd71" +dependencies = [ + "futures-util", + "log", + "rustls", + "rustls-pki-types", + "tokio", + "tokio-rustls", + "tungstenite", + "webpki-roots 0.26.11", +] + [[package]] name = "tokio-util" version = "0.7.19" @@ -4224,6 +4303,24 @@ version = "0.2.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" +[[package]] +name = "tungstenite" +version = "0.30.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e48ac77174b19c110a50ab2128b24215ac9cb40e0e12e093fb602d175c569d22" +dependencies = [ + "bytes", + "data-encoding", + "http", + "httparse", + "log", + "rand 0.10.2", + "rustls", + "rustls-pki-types", + "sha1 0.11.0", + "thiserror", +] + [[package]] name = "typed-path" version = "0.12.3" @@ -4532,6 +4629,15 @@ dependencies = [ "rustls-pki-types", ] +[[package]] +name = "webpki-roots" +version = "0.26.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "521bc38abb08001b01866da9f51eb7c5d647a19260e00054a8c7fd5f9e57f7a9" +dependencies = [ + "webpki-roots 1.0.9", +] + [[package]] name = "webpki-roots" version = "1.0.9" @@ -4886,7 +4992,7 @@ dependencies = [ "rand 0.8.7", "serde", "serde_repr", - "sha1", + "sha1 0.10.7", "static_assertions", "tokio", "tracing", diff --git a/rust/Cargo.toml b/rust/Cargo.toml index 4978212a..f940e70e 100644 --- a/rust/Cargo.toml +++ b/rust/Cargo.toml @@ -25,6 +25,7 @@ dirs = "6" domains-client = { path = "domains-client" } fancy-regex = "0.14" flate2 = { version = "1.1.9", default-features = false, features = ["rust_backend"] } +futures-util = { version = "0.3", default-features = false, features = ["std", "sink"] } globset = "0.4" open = "5" oxc_allocator = "0.143" @@ -44,6 +45,7 @@ serde_json = "1" tar = { version = "0.4.46", default-features = false } thiserror = "2" tokio = { version = "1", features = ["full"] } +tokio-tungstenite = { version = "0.30.0", features = ["connect", "handshake", "rustls-tls-webpki-roots"] } toml = "0.9" tracing = "0.1" tracing-subscriber = { version = "0.3", features = ["env-filter"] } diff --git a/rust/src/db/mod.rs b/rust/src/db/mod.rs new file mode 100644 index 00000000..186bb7ac --- /dev/null +++ b/rust/src/db/mod.rs @@ -0,0 +1,26 @@ +//! `gddy db` — access an application's database. +//! +//! Currently exposes `db tunnel`, a MySQL-over-WebSocket bridge to an app's +//! agent. The relay itself lives in [`tunnel`]; this module is wiring only. + +use cli_engine::{GroupSpec, Module, RuntimeGroupSpec, Stage}; + +mod tunnel; + +/// The `Database` module: the `db` command group. +pub fn module() -> Module { + Module::new("Database", |_ctx| { + RuntimeGroupSpec::new( + GroupSpec::new("db", "Access an application's database").with_long( + "Database access helpers for GoDaddy-hosted applications.\n\n\ + `db tunnel` opens a local TCP port and bridges raw MySQL traffic over \ + an authenticated WebSocket to the application's agent, which dials the \ + app's configured database. Point any MySQL client at the local port. \ + End-to-end MySQL authentication and TLS are preserved — the tunnel \ + forwards bytes only and never injects or inspects credentials.", + ), + ) + .with_command(tunnel::command()) + }) + .with_feature_flag("db", Stage::Experimental) +} diff --git a/rust/src/db/tunnel.rs b/rust/src/db/tunnel.rs new file mode 100644 index 00000000..153c9932 --- /dev/null +++ b/rust/src/db/tunnel.rs @@ -0,0 +1,604 @@ +//! `gddy db tunnel` — MySQL-over-WebSocket bridge to an application's agent. +//! +//! Opens a local TCP listener; for every MySQL client connection it opens one +//! WebSocket to the agent at `/apps/:id/database/tunnel` and relays raw MySQL +//! bytes in both directions (one WebSocket per TCP connection, no multiplexing). +//! The agent dials the app's configured MySQL host:port, so end-to-end MySQL +//! authentication and TLS are preserved — this client forwards bytes only and +//! never injects or inspects credentials. Progress is streamed as JSON events, +//! matching `platform app deploy`. +//! +//! Auth: the CLI mints a short-lived agent token from the hosting API +//! (`POST /v1/hosting/nodejs/apps/:id/agent-token`) using your GoDaddy OAuth +//! credential, then connects to the agent URL that call returns, sending the +//! minted token as `Authorization: Bearer`. The agent URL and token both come +//! from the service — neither is a user-supplied flag. + +use std::sync::Arc; +use std::sync::atomic::{AtomicU64, Ordering}; + +use bytes::Bytes; +use cli_engine::{CommandContext, CommandSpec, RuntimeCommandSpec, StreamSender, Tier}; +use futures_util::stream::FuturesUnordered; +use futures_util::{SinkExt, StreamExt}; +use serde_json::{Value, json}; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; +use tokio::net::{TcpListener, TcpStream}; +use tokio_tungstenite::tungstenite::Message; +use tokio_tungstenite::tungstenite::client::IntoClientRequest; +use tokio_tungstenite::tungstenite::http::{HeaderValue, header::AUTHORIZATION}; +use tokio_tungstenite::tungstenite::protocol::WebSocketConfig; +use tokio_tungstenite::{MaybeTlsStream, WebSocketStream, connect_async_with_config}; + +use crate::application::client::api_url_for_env; +use crate::error::GddyError; +use crate::hosting::nodejs::client::HostingClient; +use crate::scopes::HOSTING_DEPLOY_EXECUTE as DEPLOY_EXECUTE; + +/// A connected agent WebSocket (TLS for `wss`, plain for `ws`). +type AgentSocket = WebSocketStream>; + +/// Bytes read from the local MySQL client per WebSocket frame. Kept well under +/// the agent's 32 MiB frame cap; MySQL reassembles its own packets from the +/// byte stream regardless of framing. +const UP_CHUNK_BYTES: usize = 64 * 1024; + +/// Ceiling for a single inbound WebSocket frame from the agent. Matches the +/// agent's documented 32 MiB frame cap so a large downstream result frame is +/// not rejected by tungstenite's 16 MiB default and torn down mid-query. +const MAX_AGENT_FRAME_BYTES: usize = 32 * 1024 * 1024; + +/// How often to flush the WebSocket sink while a connection is idle, so the +/// agent's ping keepalive is answered (it pings every 30s and drops the tunnel +/// if a pong misses the next tick). Comfortably inside that window. +const FLUSH_INTERVAL: std::time::Duration = std::time::Duration::from_secs(10); + +#[derive(Debug, Clone, clap::Args)] +struct TunnelArgs { + /// Application/site id — the app whose database to tunnel to. The CLI mints + /// an agent token for this app and connects to its assigned agent. + #[arg(long = "app-id", value_name = "APP_ID")] + app_id: String, + + /// Local TCP port MySQL clients connect to. + #[arg(long, value_name = "PORT", default_value_t = 3306)] + port: u16, + + /// Local interface to bind. Defaults to loopback. + #[arg(long = "listen-host", value_name = "HOST", default_value = "127.0.0.1")] + listen_host: String, +} + +pub(super) fn command() -> RuntimeCommandSpec { + RuntimeCommandSpec::new_typed_streaming::( + CommandSpec::from_args::( + "tunnel", + "Bridge a local port to an app's MySQL over the agent WebSocket", + ) + .with_long( + "Open a local TCP listener and forward raw MySQL traffic to an \ + application's agent over a WebSocket. Each client connection gets \ + its own WebSocket to the agent's `/apps//database/tunnel` \ + endpoint; the agent dials the app's configured database. Point any \ + MySQL client at the local port, for example:\n\n\ + \tmysql -h 127.0.0.1 -P 3306 -u -p\n\n\ + MySQL authentication and TLS are negotiated end-to-end with the \ + database — the tunnel forwards bytes only and never injects or \ + inspects credentials. The CLI authorizes with your GoDaddy \ + credentials and connects to the app's assigned agent automatically. \ + Runs until interrupted (Ctrl-C).", + ) + .with_system("database") + .with_tier(Tier::Mutate) + .with_scopes(&[DEPLOY_EXECUTE]), + |ctx, args: TunnelArgs, sender: StreamSender| async move { + run_tunnel(&ctx, args, &sender).await + }, + ) +} + +/// Build the terminal `{"type":"error",...}` event, reusing +/// `cli_engine::build_error_envelope` so `code`/`message`/`fix` match what a +/// non-streaming command would render for the same error. +fn tunnel_error_event(err: &cli_engine::CliCoreError) -> Value { + let envelope = cli_engine::build_error_envelope(err, "database"); + let (code, message) = envelope + .error + .map(|e| (e.code, e.message)) + .unwrap_or_else(|| ("ERROR".to_owned(), err.to_string())); + let mut event = json!({ + "type": "error", + "ok": false, + "error": { "code": code, "message": message }, + "next_actions": [], + }); + if let Some(fix) = envelope.fix.filter(|f| !f.is_empty()) { + event["fix"] = json!(fix); + } + event +} + +/// Emit the terminal error event, then return the error so the handler can fail +/// the run via `?` — every failure path produces exactly one terminal line. +async fn fail(sender: &StreamSender, err: cli_engine::CliCoreError) -> cli_engine::CliCoreError { + sender.send(tunnel_error_event(&err)).await; + err +} + +async fn run_tunnel( + ctx: &CommandContext, + args: TunnelArgs, + sender: &StreamSender, +) -> cli_engine::Result<()> { + // Validate the app id before it is interpolated into any URL (the + // agent-token request path and the WebSocket route), so a malformed value + // fails fast with a clean error instead of silently corrupting a request + // path and surfacing a confusing 404/405 from the hosting API. + if let Err(e) = validate_app_id(&args.app_id) { + return Err(fail(sender, e.into_cli_error()).await); + } + + // Mint an agent token (and learn the agent's URL) before binding a port, so + // an auth or lookup failure fails fast with a single terminal error line. + sender + .send(json!({ "type": "step", "name": "authorize", "status": "started" })) + .await; + let (agent_url, token) = match mint_agent_token(ctx, &args.app_id).await { + Ok(pair) => pair, + Err(e) => return Err(fail(sender, e).await), + }; + sender + .send(json!({ "type": "step", "name": "authorize", "status": "completed" })) + .await; + + let ws_url = match build_tunnel_ws_url(&agent_url, &args.app_id) { + Ok(url) => url, + Err(e) => return Err(fail(sender, e.into_cli_error()).await), + }; + + let bind_addr = format!("{}:{}", args.listen_host, args.port); + let listener = match TcpListener::bind(&bind_addr).await { + Ok(listener) => listener, + Err(e) => { + let err = GddyError::network(format!("could not bind {bind_addr}: {e}")) + .with_fix("Pick a free port with --port, or stop the process already using it.") + .into_cli_error(); + return Err(fail(sender, err).await); + } + }; + let local_addr = listener + .local_addr() + .map(|a| a.to_string()) + .unwrap_or_else(|_| bind_addr.clone()); + + sender + .send(json!({ + "type": "listening", + "address": local_addr, + "agent": ws_url, + "appId": args.app_id, + })) + .await; + sender + .send(json!({ + "type": "hint", + "message": format!( + "Connect a MySQL client: mysql -h {} -P {} -u -p", + args.listen_host, args.port + ), + })) + .await; + + let token = token.as_str(); + let mut conns = FuturesUnordered::new(); + let mut accepted: u64 = 0; + + // Register the Ctrl-C handler once and poll the same future each iteration; + // recreating it per loop turn re-registers the signal handler needlessly. + let shutdown = tokio::signal::ctrl_c(); + tokio::pin!(shutdown); + + loop { + tokio::select! { + accept = listener.accept() => { + match accept { + Ok((stream, peer)) => { + accepted += 1; + conns.push(handle_connection(accepted, stream, peer.to_string(), &ws_url, token, sender)); + } + Err(e) => { + sender + .send(json!({ "type": "warning", "message": format!("accept failed: {e}") })) + .await; + // Back off briefly so a persistent accept error (e.g. the + // process is out of file descriptors) cannot spin into a + // tight loop that floods warnings and pins a core. + tokio::time::sleep(std::time::Duration::from_millis(50)).await; + } + } + } + // Drive in-flight connections; each emits its own close event. + _ = conns.next(), if !conns.is_empty() => {} + _ = &mut shutdown => { + sender + .send(json!({ "type": "step", "name": "shutdown", "status": "started" })) + .await; + break; + } + } + } + + // Dropping `conns` cancels any still-open relays (closing their sockets); + // that is the expected outcome of an interactive Ctrl-C. + sender + .send(json!({ + "type": "result", + "ok": true, + "result": { "appId": args.app_id, "connections": accepted }, + "next_actions": [], + })) + .await; + Ok(()) +} + +/// Mint a short-lived agent token for `app_id` via the hosting API and return +/// `(agent_url, token)`. Uses the CLI's OAuth credential stepped up to the +/// deploy-execute scope — the same authorization `hosting nodejs deployment +/// publish` requires — so no separate site JWT is needed. +async fn mint_agent_token( + ctx: &CommandContext, + app_id: &str, +) -> cli_engine::Result<(String, String)> { + let required = vec![DEPLOY_EXECUTE.to_owned()]; + let token = ctx.credential_with_scopes(&required).await?.token; + let base_url = api_url_for_env(&ctx.middleware.env)?; + let client = HostingClient::new(base_url, token); + let resp = client + .get_agent_token(app_id) + .await + .map_err(|e| GddyError::from(e).into_cli_error())?; + let agent_url = field_str(&resp, "agentUrl")?; + let token = field_str(&resp, "token")?; + Ok((agent_url, token)) +} + +/// Pull a required string field out of the agent-token response, mapping a +/// missing or non-string value to a coded error (the service contract is broken). +fn field_str(resp: &Value, key: &str) -> cli_engine::Result { + resp.get(key) + .and_then(Value::as_str) + .map(str::to_owned) + .ok_or_else(|| { + GddyError::network(format!("hosting agent-token response missing '{key}'")) + .with_fix("Retry; if it persists, the app may not support database tunneling yet.") + .into_cli_error() + }) +} + +/// Handle one accepted MySQL client: open its own agent WebSocket and relay +/// bytes until either side closes. Emits `connection` open/close/error events; +/// a failure here never tears down the listener. +async fn handle_connection( + id: u64, + stream: TcpStream, + peer: String, + ws_url: &str, + token: &str, + sender: &StreamSender, +) { + let _ = stream.set_nodelay(true); + + let ws = match connect_agent(ws_url, token).await { + Ok(ws) => ws, + Err(e) => { + sender + .send(json!({ + "type": "connection", + "event": "error", + "id": id, + "peer": peer, + "error": e.to_string(), + })) + .await; + return; + } + }; + sender + .send(json!({ "type": "connection", "event": "open", "id": id, "peer": peer })) + .await; + + let (up, down, reason) = relay(stream, ws).await; + + sender + .send(json!({ + "type": "connection", + "event": "close", + "id": id, + "peer": peer, + "bytesUp": up, + "bytesDown": down, + "reason": reason, + })) + .await; +} + +/// Open a WebSocket to the agent's tunnel endpoint, attaching the minted token +/// as `Authorization: Bearer `. +async fn connect_agent(ws_url: &str, token: &str) -> Result { + let mut request = ws_url + .into_client_request() + .map_err(|e| GddyError::validation(format!("invalid agent URL '{ws_url}': {e}")))?; + let value: HeaderValue = format!("Bearer {token}") + .parse() + .map_err(|_| GddyError::validation("minted agent token is not a valid header value"))?; + request.headers_mut().insert(AUTHORIZATION, value); + // Raise the inbound frame limit to the agent's documented 32 MiB cap; + // tungstenite's 16 MiB default would otherwise reject a large downstream + // result frame with a Capacity error and tear the connection down mid-query. + let config = WebSocketConfig::default().max_frame_size(Some(MAX_AGENT_FRAME_BYTES)); + let (ws, _response) = connect_async_with_config(request, Some(config), false) + .await + .map_err(map_ws_err)?; + Ok(ws) +} + +/// Relay bytes between the MySQL client and the agent WebSocket until one side +/// closes. Returns `(bytes_up, bytes_down, close_reason)`. +/// +/// Each direction runs as its own future so backpressure in one never blocks +/// the other. A single shared `select!` that awaited `send`/`write_all` inline +/// would, under simultaneous bidirectional backpressure, suspend the whole loop +/// on one direction's blocked write and stop draining the other — deadlocking +/// the connection and starving the keepalive flush. Splitting the directions +/// lets each make progress (and the flush timer fire) independently. Byte +/// counters are shared and incremented only after a successful forward, so the +/// reported totals never include a chunk that failed to reach the far side. +async fn relay(stream: TcpStream, ws: AgentSocket) -> (u64, u64, &'static str) { + let (mut tcp_read, mut tcp_write) = stream.into_split(); + let (mut ws_tx, mut ws_rx) = ws.split(); + let up = Arc::new(AtomicU64::new(0)); + let down = Arc::new(AtomicU64::new(0)); + + // Upstream: MySQL client -> agent. Owns the sink, so it also flushes the + // sink periodically to push out tungstenite's auto-queued pong replies even + // while the client is idle (the agent pings every 30s and drops the tunnel + // if a pong misses the next tick). + let upstream = { + let up = Arc::clone(&up); + async move { + let mut buf = vec![0u8; UP_CHUNK_BYTES]; + let mut flush = tokio::time::interval(FLUSH_INTERVAL); + flush.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip); + flush.tick().await; // consume the immediate first tick + loop { + tokio::select! { + read = tcp_read.read(&mut buf) => match read { + Ok(0) => { + let _ = ws_tx.send(Message::Close(None)).await; + return "client-closed"; + } + Ok(n) => { + if ws_tx + .send(Message::Binary(Bytes::copy_from_slice(&buf[..n]))) + .await + .is_err() + { + return "agent-send-failed"; + } + up.fetch_add(n as u64, Ordering::Relaxed); + } + Err(_) => return "client-read-error", + }, + _ = flush.tick() => { + let _ = ws_tx.flush().await; + } + } + } + } + }; + + // Downstream: agent -> MySQL client. + let downstream = { + let down = Arc::clone(&down); + async move { + loop { + match ws_rx.next().await { + Some(Ok(Message::Binary(data))) => { + if tcp_write.write_all(&data).await.is_err() { + return "client-write-error"; + } + down.fetch_add(data.len() as u64, Ordering::Relaxed); + } + Some(Ok(Message::Close(_))) => { + let _ = tcp_write.shutdown().await; + return "agent-closed"; + } + // Ping is auto-answered by tungstenite; the pong is flushed by + // the upstream flush timer. Text/Pong/Frame are not part of the + // byte relay and are ignored. + Some(Ok(_)) => {} + Some(Err(_)) => return "agent-error", + None => { + let _ = tcp_write.shutdown().await; + return "agent-eof"; + } + } + } + } + }; + + // Stop as soon as either direction ends; dropping the other future closes + // its half of each socket, tearing the paired connection down. + let reason = tokio::select! { + r = upstream => r, + r = downstream => r, + }; + ( + up.load(Ordering::Relaxed), + down.load(Ordering::Relaxed), + reason, + ) +} + +/// Validate `--app-id` before it is interpolated into any URL. The value is +/// placed into both the agent-token request path and the WebSocket route, so +/// anything that could alter URL structure — path separators, query/fragment +/// markers, whitespace, percent-escapes — must be rejected up front rather than +/// silently corrupting a request. App ids are short alphanumeric slugs, so an +/// allowlist of letters, digits, `-` and `_` is both sufficient and safe. +fn validate_app_id(app_id: &str) -> Result<(), GddyError> { + if app_id.is_empty() { + return Err(GddyError::validation("--app-id must not be empty")); + } + if !app_id + .chars() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, '-' | '_')) + { + return Err(GddyError::validation( + "--app-id may contain only letters, digits, '-' and '_'", + )); + } + Ok(()) +} + +/// Turn an agent base URL + app id into the WebSocket tunnel URL, mapping +/// `http`→`ws` and `https`→`wss` and replacing the path with the tunnel route. +fn build_tunnel_ws_url(agent_url: &str, app_id: &str) -> Result { + validate_app_id(app_id)?; + let parsed = url::Url::parse(agent_url).map_err(|e| { + GddyError::network(format!( + "hosting service returned an invalid agent URL '{agent_url}': {e}" + )) + })?; + let ws_scheme = match parsed.scheme() { + "http" | "ws" => "ws", + "https" | "wss" => "wss", + other => { + return Err(GddyError::network(format!( + "agent URL has an unsupported scheme '{other}': expected http, https, ws, or wss" + ))); + } + }; + let host = parsed + .host_str() + .ok_or_else(|| GddyError::network(format!("agent URL '{agent_url}' has no host")))?; + let authority = match parsed.port() { + Some(port) => format!("{host}:{port}"), + None => host.to_owned(), + }; + Ok(format!( + "{ws_scheme}://{authority}/apps/{app_id}/database/tunnel" + )) +} + +/// Map a tungstenite connect error to a coded [`GddyError`] with a status-aware +/// recovery hint (the agent's handshake failures are the common case). +fn map_ws_err(err: tokio_tungstenite::tungstenite::Error) -> GddyError { + use tokio_tungstenite::tungstenite::Error as WsErr; + match err { + WsErr::Http(response) => { + let status = response.status(); + let hint = match status.as_u16() { + 401 => { + "The agent rejected the token. Retry to mint a fresh one; if it persists, contact support." + } + 403 => { + "The token is not authorized for this app. Confirm --app-id is one of your applications." + } + 404 => { + "Tunnel endpoint not found on the agent. Confirm the app has database tunneling enabled." + } + 429 => "Too many active tunnels for this app. Close some and retry.", + 503 => "Agent is draining or shutting down. Retry shortly.", + _ => "Retry the command; if it persists, contact support.", + }; + GddyError::network(format!("agent handshake failed: HTTP {status}")).with_fix(hint) + } + other => GddyError::network(format!("could not connect to agent: {other}")) + .with_fix("Check your network connection and retry."), + } +} + +#[cfg(test)] +mod tests { + use super::build_tunnel_ws_url; + + #[test] + fn builds_ws_url_from_http_agent_with_port() { + let url = build_tunnel_ws_url("http://127.0.0.1:4000", "abcdef1234").expect("valid url"); + assert_eq!(url, "ws://127.0.0.1:4000/apps/abcdef1234/database/tunnel"); + } + + #[test] + fn builds_wss_url_from_https_agent_without_port() { + let url = + build_tunnel_ws_url("https://agent.host.example", "abcdef1234").expect("valid url"); + assert_eq!( + url, + "wss://agent.host.example/apps/abcdef1234/database/tunnel" + ); + } + + #[test] + fn preserves_ws_and_wss_schemes() { + assert!( + build_tunnel_ws_url("ws://localhost:8080", "abcdef1234") + .expect("valid") + .starts_with("ws://") + ); + assert!( + build_tunnel_ws_url("wss://host.example", "abcdef1234") + .expect("valid") + .starts_with("wss://") + ); + } + + #[test] + fn ignores_any_path_on_agent_url() { + // Only scheme://authority is used; any path on the agent URL is replaced. + let url = + build_tunnel_ws_url("https://host.example/ignored/path", "abcdef1234").expect("valid"); + assert_eq!(url, "wss://host.example/apps/abcdef1234/database/tunnel"); + } + + #[test] + fn rejects_unsupported_scheme() { + assert!(build_tunnel_ws_url("ftp://host", "abcdef1234").is_err()); + } + + #[test] + fn rejects_missing_host() { + assert!(build_tunnel_ws_url("http://", "abcdef1234").is_err()); + } + + #[test] + fn rejects_bad_app_id() { + assert!(build_tunnel_ws_url("http://host:3306", "").is_err()); + assert!(build_tunnel_ws_url("http://host:3306", "a/b").is_err()); + } + + #[test] + fn rejects_app_id_with_url_metacharacters() { + // Characters that would alter URL structure must be rejected before the + // id reaches the agent-token path or the WebSocket route. + for bad in ["abc#x", "abc?x", "abc x", "abc%2f", "a.b", "a:b"] { + assert!( + build_tunnel_ws_url("http://host:3306", bad).is_err(), + "expected {bad:?} to be rejected" + ); + } + } + + #[test] + fn tunnel_error_event_carries_code_and_fix() { + let err = crate::error::GddyError::network("boom") + .with_fix("do the thing") + .into_cli_error(); + let event = super::tunnel_error_event(&err); + assert_eq!(event["type"], "error"); + assert_eq!(event["ok"], false); + assert_eq!(event["error"]["code"], crate::error::codes::NETWORK_ERROR); + assert_eq!(event["error"]["message"], "boom"); + assert_eq!(event["fix"], "do the thing"); + } +} diff --git a/rust/src/hosting/nodejs/client.rs b/rust/src/hosting/nodejs/client.rs index f86b8a6a..29aff7fe 100644 --- a/rust/src/hosting/nodejs/client.rs +++ b/rust/src/hosting/nodejs/client.rs @@ -168,6 +168,19 @@ impl HostingClient { .await } + /// Mint a short-lived agent token for the app and return the agent's + /// assigned URL alongside it. Response shape: `{ agentUrl, token, expires? }`. + /// Requires the same `hosting.paas.deploy:execute` scope as `publish_app`. + pub async fn get_agent_token(&self, app_id: &str) -> Result { + self.send_json( + Method::POST, + &format!("/apps/{app_id}/agent-token"), + &[], + Some(json!({})), + ) + .await + } + pub async fn get_app_status(&self, app_id: &str) -> Result { self.send_json(Method::GET, &format!("/apps/{app_id}/status"), &[], None) .await @@ -550,6 +563,32 @@ mod tests { assert_eq!(body["deploymentId"], "dep-1"); } + #[tokio::test] + async fn get_agent_token_posts_empty_body_and_returns_url_and_token() { + let server = MockServer::start_async().await; + let mock = server + .mock_async(|when, then| { + when.method(POST) + .path("/v1/hosting/nodejs/apps/app-1/agent-token") + .header("authorization", "Bearer test-token") + .json_body(json!({})); + then.status(200).json_body(json!({ + "agentUrl": "https://app-1.agent.example", + "token": "minted-agent-jwt" + })); + }) + .await; + + let body = client(&server.base_url()) + .get_agent_token("app-1") + .await + .expect("get agent token"); + + mock.assert_async().await; + assert_eq!(body["agentUrl"], "https://app-1.agent.example"); + assert_eq!(body["token"], "minted-agent-jwt"); + } + #[tokio::test] async fn get_git_import_status_sends_job_id() { let server = MockServer::start_async().await; diff --git a/rust/src/main.rs b/rust/src/main.rs index aef05f4f..eec7a7fd 100644 --- a/rust/src/main.rs +++ b/rust/src/main.rs @@ -4,6 +4,7 @@ mod application; mod auth; mod config; mod contacts; +mod db; mod dns; mod domain; mod email; @@ -39,6 +40,7 @@ use crate::next_action::next_action; pub(crate) fn all_modules() -> Vec { vec![ api_explorer::module(), + db::module(), dns::module(), domain::module(), email::module(), @@ -380,6 +382,46 @@ mod tests { } } + /// The `db` group is `Stage::Experimental`: hidden at the GA default, and + /// once revealed it publishes `db tunnel` with its full flag set. Guards the + /// `main.rs` wiring and that `TunnelArgs` parses into the command spec. + #[tokio::test] + async fn db_tunnel_is_gated_and_exposes_its_flags_when_revealed() { + let hidden = Cli::new( + CliConfig::new("gddy", "GoDaddy developer CLI", "gddy") + .with_min_stage(Stage::Ga) + .with_modules(super::all_modules()), + ); + let output = hidden.run(["gddy", "db", "--help"]).await; + assert_ne!( + output.exit_code, 0, + "db should stay hidden at the Ga default: {}", + output.rendered + ); + + let revealed = Cli::new( + CliConfig::new("gddy", "GoDaddy developer CLI", "gddy") + .with_min_stage(Stage::Experimental) + .with_modules(super::all_modules()), + ); + let output = revealed.run(["gddy", "db", "tunnel", "--help"]).await; + assert_eq!(output.exit_code, 0, "{}", output.rendered); + for flag in ["--app-id", "--port", "--listen-host"] { + assert!( + output.rendered.contains(flag), + "db tunnel help should list {flag:?}: {}", + output.rendered + ); + } + for gone in ["--agent-url", "--jwt"] { + assert!( + !output.rendered.contains(gone), + "db tunnel help should no longer list the local/testing flag {gone:?}: {}", + output.rendered + ); + } + } + // `--env` actually re-routing command execution to the targeted // environment (DEVEX-721's `cli-smoke` env-override parity item) is // already covered end-to-end per-command — see