Skip to content

docs(raw): publish 0.1.1 benchmark results - #15

Merged
bproject07 merged 1 commit into
mainfrom
docs/codex-raw-0.1.1-benchmarks
Jul 29, 2026
Merged

docs(raw): publish 0.1.1 benchmark results#15
bproject07 merged 1 commit into
mainfrom
docs/codex-raw-0.1.1-benchmarks

Conversation

@bproject07

Copy link
Copy Markdown
Owner

Summary

  • publish a second reproducible benchmark bound to immutable codex-raw-v0.1.1
  • add 20 sanitized per-sample measurements plus exact release, tag, asset, binary, harness, environment, and method provenance
  • update the benchmark overview and exact-tag reproduction commands

Run

  • exact GitHub Windows x64 release asset; no local compilation
  • gpt-5.4-mini, public prompt hello
  • 2 warm-up turns and 20 measured turns (10 app-server + 10 api-server)
  • sequential, no retries, no failures or timeouts

Privacy and interpretation

The v2 samples retain only order, mode, monotonic TTFT/total latency, input-token count, and output length. They contain no response text, output fingerprint, credential, account identifier, hostname, private address, or absolute local path. Unsalted output hashes were intentionally omitted because low-entropy outputs can be reconstructed from them.

The observed VM wall-clock interval is inconsistent with the summed monotonic samples by 242.5499 ms. This is recorded explicitly; no cause is asserted, and wall time is not used for latency calculations.

The two dated release runs are not presented as a controlled release-to-release performance comparison.

Verification

  • independent recomputation of all medians, p90/p95, min/max, means, counts, ordering, and token invariants
  • independent GitHub release/tag/asset/SHA256SUMS/binary/harness provenance audit
  • staged JSON/hash binding and secret/privacy scan
  • both PowerShell reproduction blocks parse and verify the exact tag commit and harness Git blob

@bproject07
bproject07 merged commit 2a88395 into main Jul 29, 2026
1 check passed
@bproject07
bproject07 deleted the docs/codex-raw-0.1.1-benchmarks branch July 29, 2026 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant