libs/libc/machine/risc-v: Correct RISC-V optimized functions. - #19856
Merged
Conversation
The word loop walks src to a register boundary and then stores a whole register at a time to dst, but nothing establishes that dst is on a boundary too. Where the two pointers disagree about where a boundary falls, every store in that loop is misaligned. The base ISA does not require misaligned stores to be supported. Where firmware emulates them each store traps into machine mode, and where nothing emulates them the store faults, so this is not only a question of speed. Measured on a 1.4 GHz rv64 that emulates them, with a 32 KB string whose src and dst are misaligned by different amounts: generic C 410.4 MB/s this file 7.5 MB/s which is around 178 cycles per byte, flat from 512 bytes to 32 KB. Test the two pointers against each other before going wide, as arch_strcpy.S already does. Pointers that agree still reach the word loop, since walking src to a boundary walks dst to one as well; pointers that disagree take the byte path, where no single boundary serves both. After the change the misaligned case runs at 490 MB/s and the aligned rates are unchanged. The measurements come from the benchmark in apache/nuttx-apps#3706. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
memcmp, strncmp and strcmp reach their word loops only when both pointers
are already on a register boundary:
or t0, a0, a1
andi t0, t0, SZREG-1
That asks more than the loops need. They load from the two pointers at
the same boundary, so what matters is that the two agree about where a
boundary falls, not that either is already on one. A pair offset by the
same amount can be walked up to the boundary a byte at a time and
compared a register at a time from there.
The union also holds far less often than the difference. For arbitrary
pointers on RV64 it is true about one time in 64 against one in eight,
and the case it rejects, two strings carved out of the same buffer, is
the common one.
Test the difference of the pointers, and walk to the boundary first.
arch_strcpy.S and arch_memcpy.S already do this. Keeping every access
aligned is not only faster here: the base ISA does not require misaligned
loads and stores to be supported at all, so a routine in a machine
directory cannot assume one will work, whatever it costs.
Measured on a 1.4 GHz rv64, source and destination misaligned by one:
before after
memcmp 32K 34.4 458.0 MB/s
strncmp 32K 32.4 253.0 MB/s
strcmp 32K 41.0 280.0 MB/s
Each of those was the rate of the byte loop the word loop was meant to
replace. Pointers that genuinely disagree still take the byte loop, and
the aligned rates are unchanged.
The measurements come from the benchmark in apache/nuttx-apps#3706.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
This was referenced Aug 15, 2026
xiaoxiang781216
approved these changes
Aug 15, 2026
Fishwaldo
added a commit
to Fishwaldo/nuttx
that referenced
this pull request
Aug 16, 2026
Both boards used the C string and memory routines while the architecture's hand written ones sat unused beside them. A 64 bit core with a filesystem, a network stack and a display above it spends a great deal of its time in these functions, and the assembly moves a register at a time rather than a byte. RISCV_STRING_FUNCTION selects the whole set, so one symbol covers memcpy, memset, memmove and the string routines together. The generic memset tuning options go with it, since the C memset they tune is no longer built. These routines need the alignment fixes in apache#19856 and apache#19857 to be correct on misaligned pointers. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
Fishwaldo
added a commit
to Fishwaldo/nuttx
that referenced
this pull request
Aug 16, 2026
Both boards used the C string and memory routines while the architecture's hand written ones sat unused beside them. A 64 bit core with a filesystem, a network stack and a display above it spends a great deal of its time in these functions, and the assembly moves a register at a time rather than a byte. RISCV_STRING_FUNCTION selects the whole set, so one symbol covers memcpy, memset, memmove and the string routines together. The generic memset tuning options go with it, since the C memset they tune is no longer built. These routines need the alignment fixes in apache#19856 and apache#19857 to be correct on misaligned pointers. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
jerpelea
approved these changes
Aug 17, 2026
xiaoxiang781216
pushed a commit
that referenced
this pull request
Aug 18, 2026
Both boards used the C string and memory routines while the architecture's hand written ones sat unused beside them. A 64 bit core with a filesystem, a network stack and a display above it spends a great deal of its time in these functions, and the assembly moves a register at a time rather than a byte. RISCV_STRING_FUNCTION selects the whole set, so one symbol covers memcpy, memset, memmove and the string routines together. The generic memset tuning options go with it, since the C memset they tune is no longer built. These routines need the alignment fixes in #19856 and #19857 to be correct on misaligned pointers. Assisted-by: Claude:claude-opus-5 Signed-off-by: Justin Hammond <justin@dynam.ac>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four functions in the RISC-V machine directory decide whether to work a
register at a time by asking whether both pointers are already on a boundary.
That is the wrong question, and in one case it produces a store that the ISA
does not guarantee will work at all.
strlcpyaligns only its source and then stores a whole register at a time toa destination that is aligned by luck.
memcmp,strncmpandstrcmptestorof the two pointers, so a pair that is equally misaligned falls back tothe byte loop the word loop exists to replace.
Two commits: the
strlcpystore, then the three compares.Where this comes from
I opened #19735 with C implementations of these routines, built around
testing whether two pointers agree about where a boundary falls rather than
whether either is already on one. While that was in review, #19781 and #19782
landed assembly implementations covering a wider set of functions, so I have
dropped my versions; the assembly is the better base and is faster than my C
in most cases.
This PR carries over the one idea from #19735 that did not make it across, and
applies it to the code that is now in tree. It is not a re-run of that review.
I will close #19735 once this lands; the part of it that belongs in the shared
BSD implementation rather than an arch directory, which is what
@xiaoxiang781216 asked for there, follows as its own PR.
Why it matters more than a missed optimisation
Misaligned access is not guaranteed on RISC-V. The base ISA permits it to be
unsupported, and implementations differ:
So a routine in a machine directory cannot assume a misaligned store will
work, whatever it might cost.
arch_strcpy.Sandarch_memcpy.Salready takethis view.
arch_strlcpy.Sdoes not.This is also why QEMU is a poor place to measure or test it. QEMU executes
misaligned accesses natively at full speed, so the
strlcpydefect isinvisible there: correct results, no penalty, nothing to see. Every number
below is from silicon.
Measured
EIC7700X, rv64 at 1.4 GHz,
CONFIG_RISCV_STRING_FUNCTION=y, 32 KB operands,taken with the benchmark in apache/nuttx-apps#3706.
genericis the sameboard with the machine directory disabled, included so the byte-loop rate is
visible.
Source and destination misaligned by different amounts, which is the case
strlcpygets wrong:7.5 MB/s is about 178 cycles per byte, flat from 512 bytes to 32 KB, which is
what a trapped and emulated store costs on this part. It is 55x slower than
the generic C it replaced.
Source and destination equally misaligned, which is the case the compares
reject:
Each
beforefigure is the generic rate, so the word loops were not beingentered at all.
Both aligned, to show the guard costs nothing where it does not fire:
Pointers that genuinely disagree still take the byte loop. No unaligned access
is introduced anywhere.
Testing
testing/libc/arch_libcunder qemu rv64, all functions passbefore and after. The
strlcpydefect is a performance defect on the partsthat emulate, so no correctness test can catch it, and none did.
against every source and destination alignment pair. An aligned measurement
at a single size cannot see any of this.