NVIDIA · varshaprasad96 · Sep 3, 2026
@@ -310,6 +310,82 @@ The storage schema is intentionally narrow:
 | `created_at_ms` and `updated_at_ms` | Gateway timestamps used for ordering and list output. |
 | `labels` | JSON object carrying Kubernetes-style object labels for filtering and organization. |
 
+### Protobuf API and storage boundaries
+
+Public RPC contracts and durable protobuf formats have separate ownership. The
+`openshell.v1.OpenShell` service currently has 73 RPCs and
+`openshell.inference.v1.Inference` has four. Their request and response roots,
+streaming flags, and transitive message closure come from the public descriptor
+set generated by `openshell-core`; a fingerprint test in `openshell-server`
+requires this inventory to be reviewed whenever it changes. Compute-driver,
+credential-driver, gateway-interceptor, and supervisor-middleware services are
+compiled contracts for internal extension boundaries, not public gateway RPCs.
+The current public inventory has 77 methods, 280 messages, and 12 enums
+(`88792324febbe48e2b311f21d2541566f100762324dbca45bce3fd18852417ca`).
+
+Storage-only messages live in the private, versioned
+`openshell.storage.v1` package under `crates/openshell-server/proto`. The server
+generates these types separately, so the public descriptor set and the Rust,
+Go, Python, and TypeScript client generation inputs do not advertise them.
+
+| Storage classification | Protobuf messages | Durable use |
+|---|---|---|
+| Encoded storage roots | `StoredProviderCredentialRefreshState`, `StoredProviderProfile`, `PolicyRevisionPayload`, `DraftChunkPayload` | Complete protobuf payload stored in an object row or a scoped policy row. |
+| Nested storage-only type | `StoredRefreshMaterialDeletion` | Repeated child records inside provider refresh state. |
+| SQL materializations | `StoredPolicyRevision`, `StoredDraftChunk` | Server-only typed results assembled from indexed columns and decoded payloads; not public RPC messages. |
+| Public messages used directly as encoded storage roots | `Sandbox`, `SandboxWorkloadTemplate`, `Provider`, `Workspace`, `WorkspaceMember`, `SshSession`, `ServiceEndpoint`, `InferenceRoute` | The generated public type is also the persisted payload. `SshSession` and `InferenceRoute` are not in the current public RPC message closure. |
+| Embedded encoded root | `SandboxPolicy` | Stored in policy rows and inside the JSON settings envelope. |
+
+The 13 encoded durable roots above have a closure of 83 messages and eight
+enums (`29d46ae84cdae41d67a1650bdf2c7691088799283c0b727970782a36f29dc4d2`).
+Its intersection with the public RPC closure contains 71 messages and eight
+enums (`05add438ba041defc98d791038ae593d3f09352677cae43f2276d494205ce415`).
+The descriptor-derived test owns these full inventories; the tables here record
+the reviewed roots and classifications.
+
+| Dual-purpose encoded root | Current decision |
+|---|---|
+| `Sandbox` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `SandboxWorkloadTemplate` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `Provider` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `Workspace` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `WorkspaceMember` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `SshSession` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `ServiceEndpoint` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `InferenceRoute` | Defer a storage twin; govern its complete dependency closure as durable. |
+| `SandboxPolicy` | Defer a storage twin; govern its complete dependency closure as durable. |
+
+The public/storage overlap is deliberate for the current format. Storage twins
+for the public roots are deferred: introducing them would require a broad
+conversion boundary, and Prost does not retain unknown fields through a
+decode-and-reencode conversion. Each root therefore carries a reviewed decision
+to remain dual-purpose, and its complete transitive dependency closure is also
+a durable format. Important embedded dependencies include `ObjectMeta`,
+`ProviderProfile`, `CredentialHandle`, `SandboxPolicy`, and
+`NetworkPolicyRule`. Global and sandbox settings additionally store an encoded
+`SandboxPolicy` inside their JSON envelope.
+
+Public API compatibility and storage compatibility are reviewed independently:
+
+- Public compatibility is evaluated from public service descriptors and SDK
+  generation inputs. Storage-only packages must never enter that closure.
+- `openshell.storage.v1` is frozen. Its test fingerprint covers message names,
+  field numbers, cardinality, scalar wire types, referenced types, map-entry
+  shapes, and optional presence. Keep its decoder available and introduce a
+  new versioned package plus an explicit migration or fallback decoder for a
+  format change; never reuse removed tags or names.
+- Checked-in synthetic byte fixtures were encoded with the former
+  `openshell.v1` declarations. Current storage types must continue to decode
+  them semantically, which proves the package move does not require a database
+  rewrite. Protobuf payload bytes do not encode a message's package name.
+- A change to a dual-purpose public message or any transitive durable
+  dependency requires both public-wire review and storage-migration review.
+  Wire-incompatible changes require a migration or fallback decoder and a
+  fixture for the earlier format.
+- Mixed-version writers are unsupported. An older Prost writer can discard
+  fields it does not know when it reads and rewrites a record, even when the
+  newer field is wire-compatible.
+
 Common resources use generic helpers that derive `object_type`, `id`, `name`,
 and labels from protobuf metadata traits before encoding the full message into
 `payload`. Policy revisions and draft policy chunks use the same table but also

@@ -9,6 +9,7 @@
 version: v2
 modules:
   - path: proto
+  - path: crates/openshell-server/proto
 lint:
   use:
     - STANDARD

@@ -7,8 +7,7 @@
 
 use crate::proto::{
     InferenceRoute, ObjectForTest, Provider, Sandbox, SandboxStatus, SandboxWorkloadTemplate,
-    ServiceEndpoint, SshSession, StoredProviderCredentialRefreshState, StoredProviderProfile,
-    Workspace, WorkspaceMember,
+    ServiceEndpoint, SshSession, Workspace, WorkspaceMember,
 };
 use std::collections::HashMap;
 
@@ -233,90 +232,6 @@ impl ObjectWorkspace for Provider {
     }
 }
 
-// Implementations for StoredProviderProfile
-impl ObjectId for StoredProviderProfile {
-    fn object_id(&self) -> &str {
-        self.metadata.as_ref().map_or("", |m| m.id.as_str())
-    }
-}
-
-impl ObjectName for StoredProviderProfile {
-    fn object_name(&self) -> &str {
-        self.metadata.as_ref().map_or("", |m| m.name.as_str())
-    }
-}
-
-impl ObjectLabels for StoredProviderProfile {
-    fn object_labels(&self) -> Option<HashMap<String, String>> {
-        self.metadata.as_ref().map(|m| m.labels.clone())
-    }
-}
-
-impl SetResourceVersion for StoredProviderProfile {
-    fn set_resource_version(&mut self, version: u64) {
-        if let Some(meta) = self.metadata.as_mut() {
-            meta.resource_version = version;
-        }
-    }
-}
-
-impl GetResourceVersion for StoredProviderProfile {
-    fn get_resource_version(&self) -> u64 {
-        self.metadata.as_ref().map_or(0, |m| m.resource_version)
-    }
-}
-
-impl ObjectWorkspace for StoredProviderProfile {
-    fn object_workspace(&self) -> &str {
-        self.metadata.as_ref().map_or("", |m| m.workspace.as_str())
-    }
-    fn requires_workspace() -> bool {
-        false
-    }
-}
-
-// Implementations for StoredProviderCredentialRefreshState
-impl ObjectId for StoredProviderCredentialRefreshState {
-    fn object_id(&self) -> &str {
-        self.metadata.as_ref().map_or("", |m| m.id.as_str())
-    }
-}
-
-impl ObjectName for StoredProviderCredentialRefreshState {
-    fn object_name(&self) -> &str {
-        self.metadata.as_ref().map_or("", |m| m.name.as_str())
-    }
-}
-
-impl ObjectLabels for StoredProviderCredentialRefreshState {
-    fn object_labels(&self) -> Option<HashMap<String, String>> {
-        self.metadata.as_ref().map(|m| m.labels.clone())
-    }
-}
-
-impl SetResourceVersion for StoredProviderCredentialRefreshState {
-    fn set_resource_version(&mut self, version: u64) {
-        if let Some(meta) = self.metadata.as_mut() {
-            meta.resource_version = version;
-        }
-    }
-}
-
-impl GetResourceVersion for StoredProviderCredentialRefreshState {
-    fn get_resource_version(&self) -> u64 {
-        self.metadata.as_ref().map_or(0, |m| m.resource_version)
-    }
-}
-
-impl ObjectWorkspace for StoredProviderCredentialRefreshState {
-    fn object_workspace(&self) -> &str {
-        self.metadata.as_ref().map_or("", |m| m.workspace.as_str())
-    }
-    fn requires_workspace() -> bool {
-        true
-    }
-}
-
 // Implementations for SshSession
 impl ObjectId for SshSession {
     fn object_id(&self) -> &str {

@@ -378,10 +378,6 @@ mod tests {
             ("openshell.v1.RevokeSshSessionRequest", "token"),
             ("openshell.v1.TcpForwardInit", "authorization_token"),
             ("openshell.v1.SshSession", "token"),
-            (
-                "openshell.v1.StoredProviderCredentialRefreshState",
-                "material",
-            ),
             ("openshell.v1.ConfigureProviderRefreshRequest", "material"),
             (
                 "openshell.v1.GetSandboxProviderEnvironmentResponse",

@@ -121,6 +121,10 @@ telemetry = ["openshell-core/telemetry"]
 bundled-z3 = ["openshell-prover/bundled-z3"]
 test-support = []
 
+[build-dependencies]
+tonic-prost-build = { workspace = true }
+protoc-bin-vendored = { workspace = true }
+
 [dev-dependencies]
 hyper-rustls = { version = "0.27", default-features = false, features = ["native-tokio", "http1", "tls12", "logging", "ring"] }
 rcgen = { version = "0.13", features = ["crypto", "pem"] }

@@ -0,0 +1,50 @@
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-License-Identifier: Apache-2.0
+
+use std::env;
+use std::path::PathBuf;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let manifest_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR")?);
+    let storage_proto_dir = manifest_dir.join("proto");
+    let public_proto_dir = manifest_dir.join("../../proto");
+    let storage_proto = storage_proto_dir.join("storage.proto");
+
+    println!("cargo:rerun-if-changed={}", storage_proto.display());
+    for imported_proto in [
+        "datamodel.proto",
+        "openshell.proto",
+        "options.proto",
+        "sandbox.proto",
+    ] {
+        println!(
+            "cargo:rerun-if-changed={}",
+            public_proto_dir.join(imported_proto).display()
+        );
+    }
+
+    // SAFETY: Build scripts run in their own single-threaded process.
+    #[allow(unsafe_code)]
+    unsafe {
+        env::set_var("PROTOC", protoc_bin_vendored::protoc_bin_path()?);
+        env::set_var("PROTOC_INCLUDE", protoc_bin_vendored::include_path()?);
+    }
+
+    let descriptor_path = PathBuf::from(env::var("OUT_DIR")?).join("storage_descriptor.bin");
+    tonic_prost_build::configure()
+        .build_server(false)
+        .build_client(false)
+        .extern_path(".openshell.v1", "::openshell_core::proto")
+        .extern_path(
+            ".openshell.datamodel.v1",
+            "::openshell_core::proto::datamodel::v1",
+        )
+        .extern_path(
+            ".openshell.sandbox.v1",
+            "::openshell_core::proto::sandbox::v1",
+        )
+        .file_descriptor_set_path(&descriptor_path)
+        .compile_protos(&[storage_proto], &[storage_proto_dir, public_proto_dir])?;
+
+    Ok(())
+}