Keep the benchmark's numbers after the job that produced them (#500) - #962
Open
Rafael-SOWNet wants to merge 1 commit into
Open
Keep the benchmark's numbers after the job that produced them (#500)#962Rafael-SOWNet wants to merge 1 commit into
Rafael-SOWNet wants to merge 1 commit into
Conversation
The Kernel Benchmark workflow ran a CPU and a RAM benchmark and then threw both away: the numbers existed only in the job log, which expires, so there was no way to see whether a figure had moved without checking out an old commit and re-running it. Two changes, and between them the results survive: - RAMUsageTest now exports a CSV. It carried [MemoryDiagnoser] but no exporter, so unlike CommonFunctionsInterVersion it wrote to the console only. - The workflow uploads both benchmarks' artifacts, with if: always() so a run that fell over part way still keeps the cases that completed -- which are the interesting ones when something has just broken. Verified by running rather than by reading the attribute: BenchmarkDotNet's ArtifactsPath names a *directory*, so the output lands in benchmark_results.csv/results/DotnetBenchmark.RAMUsageTest-report.csv, and that is what the upload glob matches. The directory is already gitignored. **This is step 1 of the issue and the only step that can be taken here.** The plan it describes pushes reports to AngouriMathLab/performance-reports and builds diagrams in AngouriMathLab/performance-reports-tools; neither repository exists. So the rest wants either those repositories created or a different destination decided, and that is a call rather than an implementation.
This was referenced Aug 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 1 of #500, and the only step that can be taken from inside this repository.
The Kernel Benchmark workflow ran a CPU benchmark and a RAM benchmark and then threw both away. The numbers existed only in the job log, which expires — so there was no way to see whether a figure had moved without checking out an old commit and re-running it by hand.
Two changes:
RAMUsageTestnow exports a CSV. It carried[MemoryDiagnoser]but no exporter, so unlikeCommonFunctionsInterVersionit wrote to the console only. Half of every run was unrecoverable by construction.if: always()— a run that fell over part way still produced numbers for the cases that completed, and those are the interesting ones when something has just broken.Verified by running it, not by reading the attribute
BenchmarkDotNet'sArtifactsPathnames a directory, not a file, despite[ArtifactsPath("./benchmark_results.csv")]reading like a filename. Running the benchmark confirms where the output actually lands:which is what the upload glob matches. Worth checking rather than assuming: a path that matches nothing uploads nothing and the job still goes green, which looks exactly like success.
if-no-files-found: warnis there for the same reason.The directory is already in
.gitignore, so nothing generated is committable.What this does not do, and why
The issue's plan pushes
result.csvtoAngouriMathLab/performance-reportsand builds diagrams inAngouriMathLab/performance-reports-tools, then publishes to the site. Neither repository exists — I checked both. So the remaining steps need either those repositories created, or a different destination decided, and that is a call rather than an implementation.What this gets you meanwhile: 90 days of downloadable, comparable CSVs per commit, which is enough to answer "did this PR move the number" without re-running anything.
🤖 Generated with Claude Code