Skip to content

libs/libc/machine/risc-v: Correct RISC-V optimized functions. - #19856

Merged
jerpelea merged 2 commits into
apache:masterfrom
Fishwaldo:upstream-riscv-strlcpy-align
Aug 17, 2026
Merged

libs/libc/machine/risc-v: Correct RISC-V optimized functions.#19856
jerpelea merged 2 commits into
apache:masterfrom
Fishwaldo:upstream-riscv-strlcpy-align

Conversation

@Fishwaldo

Copy link
Copy Markdown
Contributor

Summary

Four functions in the RISC-V machine directory decide whether to work a
register at a time by asking whether both pointers are already on a boundary.
That is the wrong question, and in one case it produces a store that the ISA
does not guarantee will work at all.

strlcpy aligns only its source and then stores a whole register at a time to
a destination that is aligned by luck. memcmp, strncmp and strcmp test
or of the two pointers, so a pair that is equally misaligned falls back to
the byte loop the word loop exists to replace.

Two commits: the strlcpy store, then the three compares.

Where this comes from

I opened #19735 with C implementations of these routines, built around
testing whether two pointers agree about where a boundary falls rather than
whether either is already on one. While that was in review, #19781 and #19782
landed assembly implementations covering a wider set of functions, so I have
dropped my versions; the assembly is the better base and is faster than my C
in most cases.

This PR carries over the one idea from #19735 that did not make it across, and
applies it to the code that is now in tree. It is not a re-run of that review.
I will close #19735 once this lands; the part of it that belongs in the shared
BSD implementation rather than an arch directory, which is what
@xiaoxiang781216 asked for there, follows as its own PR.

Why it matters more than a missed optimisation

Misaligned access is not guaranteed on RISC-V. The base ISA permits it to be
unsupported, and implementations differ:

  • where firmware emulates it, every access traps into machine mode
  • where nothing emulates it, the access faults

So a routine in a machine directory cannot assume a misaligned store will
work, whatever it might cost. arch_strcpy.S and arch_memcpy.S already take
this view. arch_strlcpy.S does not.

This is also why QEMU is a poor place to measure or test it. QEMU executes
misaligned accesses natively at full speed, so the strlcpy defect is
invisible there: correct results, no penalty, nothing to see. Every number
below is from silicon.

Measured

EIC7700X, rv64 at 1.4 GHz, CONFIG_RISCV_STRING_FUNCTION=y, 32 KB operands,
taken with the benchmark in apache/nuttx-apps#3706. generic is the same
board with the machine directory disabled, included so the byte-loop rate is
visible.

Source and destination misaligned by different amounts, which is the case
strlcpy gets wrong:

                generic     before      after
  strlcpy         410.4        7.5      490.0 MB/s

7.5 MB/s is about 178 cycles per byte, flat from 512 bytes to 32 KB, which is
what a trapped and emulated store costs on this part. It is 55x slower than
the generic C it replaced.

Source and destination equally misaligned, which is the case the compares
reject:

                generic     before      after
  memcmp           30.7       34.4      456.0 MB/s
  strncmp          32.2       32.2      254.0 MB/s
  strcmp           41.2       40.9      280.0 MB/s

Each before figure is the generic rate, so the word loops were not being
entered at all.

Both aligned, to show the guard costs nothing where it does not fire:

                 before      after
  strlcpy        2474.0     2452.0 MB/s
  memcmp          452.0      458.0 MB/s
  strncmp         266.0      256.0 MB/s
  strcmp          282.0      280.9 MB/s

Pointers that genuinely disagree still take the byte loop. No unaligned access
is introduced anywhere.

Testing

  • Correctness: testing/libc/arch_libc under qemu rv64, all functions pass
    before and after. The strlcpy defect is a performance defect on the parts
    that emulate, so no correctness test can catch it, and none did.
  • Throughput: the benchmark in testing/libc/arch_libc: Add a throughput benchmark. nuttx-apps#3706, which sweeps sizes
    against every source and destination alignment pair. An aligned measurement
    at a single size cannot see any of this.

The word loop walks src to a register boundary and then stores a whole
register at a time to dst, but nothing establishes that dst is on a
boundary too.  Where the two pointers disagree about where a boundary
falls, every store in that loop is misaligned.

The base ISA does not require misaligned stores to be supported.  Where
firmware emulates them each store traps into machine mode, and where
nothing emulates them the store faults, so this is not only a question of
speed.  Measured on a 1.4 GHz rv64 that emulates them, with a 32 KB
string whose src and dst are misaligned by different amounts:

  generic C     410.4 MB/s
  this file       7.5 MB/s

which is around 178 cycles per byte, flat from 512 bytes to 32 KB.

Test the two pointers against each other before going wide, as
arch_strcpy.S already does.  Pointers that agree still reach the word
loop, since walking src to a boundary walks dst to one as well; pointers
that disagree take the byte path, where no single boundary serves both.
After the change the misaligned case runs at 490 MB/s and the aligned
rates are unchanged.

The measurements come from the benchmark in apache/nuttx-apps#3706.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
memcmp, strncmp and strcmp reach their word loops only when both pointers
are already on a register boundary:

  or    t0, a0, a1
  andi  t0, t0, SZREG-1

That asks more than the loops need.  They load from the two pointers at
the same boundary, so what matters is that the two agree about where a
boundary falls, not that either is already on one.  A pair offset by the
same amount can be walked up to the boundary a byte at a time and
compared a register at a time from there.

The union also holds far less often than the difference.  For arbitrary
pointers on RV64 it is true about one time in 64 against one in eight,
and the case it rejects, two strings carved out of the same buffer, is
the common one.

Test the difference of the pointers, and walk to the boundary first.
arch_strcpy.S and arch_memcpy.S already do this.  Keeping every access
aligned is not only faster here: the base ISA does not require misaligned
loads and stores to be supported at all, so a routine in a machine
directory cannot assume one will work, whatever it costs.

Measured on a 1.4 GHz rv64, source and destination misaligned by one:

                    before   after
  memcmp 32K          34.4   458.0 MB/s
  strncmp 32K         32.4   253.0 MB/s
  strcmp 32K          41.0   280.0 MB/s

Each of those was the rate of the byte loop the word loop was meant to
replace.  Pointers that genuinely disagree still take the byte loop, and
the aligned rates are unchanged.

The measurements come from the benchmark in apache/nuttx-apps#3706.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
@github-actions github-actions Bot added Area: OS Components OS Components issues Size: M The size of the change in this PR is medium labels Aug 15, 2026
@github-actions

Copy link
Copy Markdown

MemBrowse Memory Report

No memory changes detected for:

Fishwaldo added a commit to Fishwaldo/nuttx that referenced this pull request Aug 16, 2026
Both boards used the C string and memory routines while the
architecture's hand written ones sat unused beside them.  A 64 bit core
with a filesystem, a network stack and a display above it spends a great
deal of its time in these functions, and the assembly moves a register at
a time rather than a byte.

RISCV_STRING_FUNCTION selects the whole set, so one symbol covers memcpy,
memset, memmove and the string routines together.  The generic memset
tuning options go with it, since the C memset they tune is no longer
built.

These routines need the alignment fixes in apache#19856 and
apache#19857 to be correct on misaligned pointers.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
Fishwaldo added a commit to Fishwaldo/nuttx that referenced this pull request Aug 16, 2026
Both boards used the C string and memory routines while the
architecture's hand written ones sat unused beside them.  A 64 bit core
with a filesystem, a network stack and a display above it spends a great
deal of its time in these functions, and the assembly moves a register at
a time rather than a byte.

RISCV_STRING_FUNCTION selects the whole set, so one symbol covers memcpy,
memset, memmove and the string routines together.  The generic memset
tuning options go with it, since the C memset they tune is no longer
built.

These routines need the alignment fixes in apache#19856 and
apache#19857 to be correct on misaligned pointers.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
@jerpelea
jerpelea merged commit 0790b52 into apache:master Aug 17, 2026
78 of 132 checks passed
xiaoxiang781216 pushed a commit that referenced this pull request Aug 18, 2026
Both boards used the C string and memory routines while the
architecture's hand written ones sat unused beside them.  A 64 bit core
with a filesystem, a network stack and a display above it spends a great
deal of its time in these functions, and the assembly moves a register at
a time rather than a byte.

RISCV_STRING_FUNCTION selects the whole set, so one symbol covers memcpy,
memset, memmove and the string routines together.  The generic memset
tuning options go with it, since the C memset they tune is no longer
built.

These routines need the alignment fixes in #19856 and
#19857 to be correct on misaligned pointers.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Justin Hammond <justin@dynam.ac>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: OS Components OS Components issues Size: M The size of the change in this PR is medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants