testing/libc/arch_libc: Add a throughput benchmark. - #3706
Merged
Conversation
Contributor
Author
|
The RISC-V series this suite was written for is now open as This PR remains independent of it: the suite is arch-neutral, passes against |
Fishwaldo
force-pushed
the
upstream-arch-libc-string
branch
from
August 9, 2026 04:33
542c585 to
555e43f
Compare
acassis
previously approved these changes
Aug 9, 2026
cederom
previously approved these changes
Aug 9, 2026
cederom
left a comment
cederom
left a comment
Contributor
There was a problem hiding this comment.
Thank you @Fishwaldo :-)
xiaoxiang781216
previously requested changes
Aug 9, 2026
Contributor
|
Looks like #3712 is a joint merge of several test suites and should be merged instead? :-) |
Contributor
|
@Fishwaldo since #3712, could you try that change and close this pr? thanks. |
Contributor
|
What's the status of this PR? Did #3712 include all the new tests that are implemented here? |
Contributor
@Fishwaldo could you check where #3712 contain all your test, if not, please apply your patch on top of #3712, thanks. |
Fishwaldo
force-pushed
the
upstream-arch-libc-string
branch
from
August 15, 2026 06:09
555e43f to
31e7dc3
Compare
Fishwaldo
changed the title
Contributor
Author
Fishwaldo
force-pushed
the
upstream-arch-libc-string
branch
from
August 15, 2026 07:44
1e02b99 to
30deac5
Compare
Contributor
Author
|
Sorry, found a few functions that were missing some correctness tests while working on the optimized fixes for riscv and the newlib variants. Added memccpy, stpncpy and strlcpy for completeness (including benchmarks) |
This was referenced Aug 15, 2026
The existing speed checks time one call at one size, 128 bytes, with both operands aligned. A machine implementation usually takes its wide path only when the pointers satisfy some alignment condition, so that single point reports the best case and says nothing about the rest of the input space. Measure the same functions across a size sweep and every source and destination alignment pair instead, plus strlcpy. On rv64 the difference this exposes is not marginal: strcpy 32768 B s+0/d+0 2938.0 MB/s strcpy 32768 B s+1/d+1 2942.0 MB/s strcpy 32768 B s+1/d+2 626.0 MB/s memcmp 32768 B s+0/d+0 412.4 MB/s memcmp 32768 B s+1/d+2 41.0 MB/s Two pointers misaligned by the same amount run at the aligned rate; misaligned by different amounts they fall to a tenth of it. Neither number is visible from an aligned measurement alone. A function with no machine implementation reports the same rate at every alignment, so the sweep also shows which of them a machine directory actually covers. Each result reports MB/s, which compares across machines, and cycles per byte where perf_gettime() is reachable from an application, both from one timed loop. strcat starts from an empty destination on each turn, since appending to the last result would grow it without bound, so its figure includes that store. It sits behind TESTING_ARCH_LIBC_BENCH, default n, because a measurement runs for a fixed interval and a full sweep takes about a minute. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
strlcpy is the one function in this directory's reach that nothing here covers, and a machine directory may override it like any other. Sweep every source and destination alignment pair against sizes 1 to 64, and for each of those every capacity from zero to one past the length. Check the return value, which is the length of src whether or not the copy fit, the truncation point, the content, that a capacity of zero writes nothing at all, and that nothing lands past the terminator. The alignment pairs are the point. An implementation that walks one of the two pointers to a boundary and then copies a register at a time is correct whenever the two agree, so a test that only ever passes matching alignments says nothing about it. The timing half is guarded. perf_gettime() is not a system call, so an application reaches it only where the C library builds its own copy or where the application and the kernel are one image; calling it unconditionally leaves the test unbuildable on a kernel build, which is where the correctness half is still wanted. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
Neither is covered here, and both are overridable, so a machine or libc implementation of either goes in unmeasured and unchecked. memccpy is checked with the search character present, where the copy stops just past it and the result points there, and absent, where the whole length is copied and the result is NULL. stpncpy is checked against every capacity from zero to four past the length, for the content, the zero padding beyond the terminator, and the returned pointer, which is the terminator when the string fits and one past the end when it does not. Both sweep all sixty four source and destination alignment pairs, and both are added to the benchmark, which now covers nineteen functions. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
Every measurement repeats until a stated interval has passed, so a clock that reads the same value twice does not slow the benchmark down, it stops it returning at all. CLOCK_MONOTONIC does not advance on every target. On qemu-intel64 it reports success and stays at zero, while CLOCK_REALTIME advances normally, and the benchmark spins in its first measurement with no output after the heading. Sample each candidate twice around a busy wait and take the first one whose reading changes. Where none does, say so and skip the timing rather than hang. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
Fishwaldo
force-pushed
the
upstream-arch-libc-string
branch
from
August 16, 2026 07:55
194d3f9 to
c915202
Compare
xiaoxiang781216
approved these changes
Aug 17, 2026
jerpelea
pushed a commit
to apache/nuttx
that referenced
this pull request
Aug 17, 2026
The word loop walks src to a register boundary and then stores a whole register at a time to dst, but nothing establishes that dst is on a boundary too. Where the two pointers disagree about where a boundary falls, every store in that loop is misaligned. The base ISA does not require misaligned stores to be supported. Where firmware emulates them each store traps into machine mode, and where nothing emulates them the store faults, so this is not only a question of speed. Measured on a 1.4 GHz rv64 that emulates them, with a 32 KB string whose src and dst are misaligned by different amounts: generic C 410.4 MB/s this file 7.5 MB/s which is around 178 cycles per byte, flat from 512 bytes to 32 KB. Test the two pointers against each other before going wide, as arch_strcpy.S already does. Pointers that agree still reach the word loop, since walking src to a boundary walks dst to one as well; pointers that disagree take the byte path, where no single boundary serves both. After the change the misaligned case runs at 490 MB/s and the aligned rates are unchanged. The measurements come from the benchmark in apache/nuttx-apps#3706. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
jerpelea
pushed a commit
to apache/nuttx
that referenced
this pull request
Aug 17, 2026
memcmp, strncmp and strcmp reach their word loops only when both pointers
are already on a register boundary:
or t0, a0, a1
andi t0, t0, SZREG-1
That asks more than the loops need. They load from the two pointers at
the same boundary, so what matters is that the two agree about where a
boundary falls, not that either is already on one. A pair offset by the
same amount can be walked up to the boundary a byte at a time and
compared a register at a time from there.
The union also holds far less often than the difference. For arbitrary
pointers on RV64 it is true about one time in 64 against one in eight,
and the case it rejects, two strings carved out of the same buffer, is
the common one.
Test the difference of the pointers, and walk to the boundary first.
arch_strcpy.S and arch_memcpy.S already do this. Keeping every access
aligned is not only faster here: the base ISA does not require misaligned
loads and stores to be supported at all, so a routine in a machine
directory cannot assume one will work, whatever it costs.
Measured on a 1.4 GHz rv64, source and destination misaligned by one:
before after
memcmp 32K 34.4 458.0 MB/s
strncmp 32K 32.4 253.0 MB/s
strcmp 32K 41.0 280.0 MB/s
Each of those was the rate of the byte loop the word loop was meant to
replace. Pointers that genuinely disagree still take the byte loop, and
the aligned rates are unchanged.
The measurements come from the benchmark in apache/nuttx-apps#3706.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
acassis
approved these changes
Aug 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rebased onto #3712 as requested, and reduced to the part #3712 does not already
cover.
The correctness suite is gone. #3712 tests all sixteen functions this PR used to
test, plus
stpcpy,strcatandstrncpythat it did not, so there is nothingleft for it to add.
What remains is the throughput measurement. #3712 times one call per function at
one size, 128 bytes, with both operands aligned. This sweeps four sizes and four
source/destination alignment pairs across seventeen functions, which is #3712's
sixteen plus
strlcpy.It sits behind
TESTING_ARCH_LIBC_BENCH, default n. Nothing changes when it isoff.
Why the alignment matrix
A machine implementation usually takes its wide path only when the pointers meet
some alignment condition, so a single aligned measurement reports the best case
and says nothing about the rest of the input space. On rv64:
Two pointers misaligned by the same amount run at the aligned rate. Misaligned by
different amounts they fall to a fifth of it. Neither number is visible from an
aligned measurement alone.
A function with no machine implementation reports the same rate at every
alignment, so the sweep also shows which functions a machine directory actually
covers.
In passing this caught a
strlcpyregression in 931d5f50d4, where the misalignedcase runs 55x slower than the generic C it replaced. I will open a separate PR
for that.
What it reports
Each measurement gives MB/s, which compares across machines, and cycles per byte
where
perf_gettime()is reachable from an application, both from one timedloop.
Testing
EIC7700X (rv64, 1.4 GHz),
CONFIG_BUILD_KERNEL, 248 measurements per run, in twoconfigurations: the generic C library, and the RISC-V assembly currently on
master. Both pass.
Note for kernel builds
perf_gettime()is not insyscall.csv, so an application cannot reach itunless the C library builds its own copy
(
CONFIG_ARCH_HAVE_PERF_EVENTS_USER_ACCESS) or the build is flat. Thisbenchmark guards for that and reports MB/s alone where the counter is out of
reach.