Skip to content

perf: avoid slow exponentiation and repeated element reads#23

Open
lpatiny wants to merge 2 commits into
mainfrom
faster-distances
Open

perf: avoid slow exponentiation and repeated element reads#23
lpatiny wants to merge 2 commits into
mainfrom
faster-distances

Conversation

@lpatiny

@lpatiny lpatiny commented Jul 26, 2026

Copy link
Copy Markdown
Member

Speeds up the hot loops of the distance and similarity functions.

Algorithmic changes

  • kumarJohnson: prod ** 1.5prod * Math.sqrt(prod). No engine specializes a 1.5 exponent, so this is the biggest win, on V8 and JavaScriptCore alike.
  • minkowski: special-case p = 1 and p = 2. V8 sends x ** 1 through generic pow at ~9x the cost of a plain read; p = 2 is worth ~1.5x when called monomorphically.
  • similarity.pearson: drop the two arrays it allocated only to mean-centre before calling cosine, folding the work into two passes.

Measured with beforeAfter.js, before → after, per element (n = 10000, Float64Array, node 24):

function before after
kumarJohnson 15.50 ns 2.41 ns 6.4x
similarity.pearson 19.06 ns 3.23 ns 5.9x
minkowski(p=1) 11.51 ns 2.02 ns 5.7x

For the other functions defining const like for example from:

  for (let i = 0; i < a.length; i++) {
    ans +=
      a[i] * Math.log((2 * a[i]) / (a[i] + b[i])) +
      b[i] * Math.log((2 * b[i]) / (a[i] + b[i]));
  }

to

 for (let i = 0; i < a.length; i++) {
    const ai = a[i];
    const bi = b[i];
    const sum = ai + bi;
    ans += ai * Math.log((2 * ai) / sum) + bi * Math.log((2 * bi) / sum);
  }

has no impact, neither with bun or with nodeJS except in the case of polymorphism (once a typed array, once a number array for example) and in this case we win a factor 1.5

But as a side observation changing the type of an array is really a bad idea and we are loosing a factor 2 in speed. I wonder if we can reach this kind of issue in some of our projects.

So practically I would still make this 'const' definition even if we win speed only in case of polymorphism. @targos WDYT ?

@lpatiny
lpatiny force-pushed the faster-distances branch from 8bd41e7 to ee5f0a2 Compare July 26, 2026 19:33
@lpatiny lpatiny changed the title Faster distances perf: avoid slow exponentiation and repeated element reads Jul 26, 2026
lpatiny added 2 commits July 26, 2026 21:42
Built on benchmark.js, the library the other mljs repositories use:

- all.js lists the cost of every exported function
- beforeAfter.js compares two implementations of the same function,
  both copied into one process so they see identical data
- exponentiation.js isolates the cost of the `**` operator
- elementReads.js isolates the cost of repeated `a[i]` reads
- equivalence.js checks `x ** 1.5` against `x * Math.sqrt(x)`

Baseline cost of the current implementations, measured with beforeAfter.js
on node 24 (n = 10000, Float64Array). These are per-element costs of each
function, not speedups:

  topsoe              21.99 ns/element
  similarity.pearson  19.06 ns/element
  kumarJohnson        15.50 ns/element
  minkowski(p=1)      11.51 ns/element
  squared              4.51 ns/element
  cosine               4.38 ns/element
  pearson              3.74 ns/element
  clark                3.22 ns/element

The next commit optimizes these and reports the same measurements again.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three algorithmic changes, each large and stable across engines:

- kumarJohnson: `prod ** 1.5` -> `prod * Math.sqrt(prod)`. No engine
  specializes a 1.5 exponent, so this is the single biggest win.
- minkowski: special-case p = 1 and p = 2. V8 runs `x ** 1` through generic
  pow at ~9x the cost of a plain read; p = 2 is worth ~1.5x.
- similarity.pearson: drop the two arrays it allocated only to mean-centre
  before calling cosine, and fold the work into two passes.

Measured with beforeAfter.js, before -> after (n = 10000, Float64Array,
node 24), against the baseline recorded in the previous commit:

  kumarJohnson        15.50 -> 2.41 ns/element    6.4x faster
  similarity.pearson  19.06 -> 3.23 ns/element    5.9x faster
  minkowski(p=1)      11.51 -> 2.02 ns/element    5.7x faster
  squared              4.51 -> 2.22 ns/element    2.0x faster
  cosine               4.38 -> 2.19 ns/element    2.0x faster
  pearson              3.74 -> 2.14 ns/element    1.7x faster
  clark                3.22 -> 2.17 ns/element    1.5x faster
  topsoe              21.99 -> 17.21 ns/element   1.3x faster

The remaining change caches repeated `a[i]` reads in locals. Its value is
much smaller and depends on what the load site actually observes at run
time, not on any TypeScript type: V8 often eliminates the repeated loads by
itself. elementReads.js measures anywhere from 1.00x when a site only ever
sees one Float64Array shape up to ~1.5x once it has seen several array
kinds, and the exact figure moves with how the harness is built. The rows
above are the higher end of that range because beforeAfter.js exercises
every function with both array kinds in one process. It is never a
regression, so it stays, but it is not a reliable 2x.

JavaScriptCore eliminates those loads outright: on bun the caching is
1.00x everywhere and only kumarJohnson (4x) and similarity.pearson (6x)
improve.

kumarJohnson moves by at most 1 ulp, and `(-Infinity) ** 1.5` is Infinity
where `-Infinity * Math.sqrt(-Infinity)` is NaN; both are outside the
non-negative domain the metric is defined on. Its test now uses toBeCloseTo.

ml-array-mean is no longer used.

Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lpatiny
lpatiny force-pushed the faster-distances branch from ee5f0a2 to 6f39522 Compare July 26, 2026 19:43
@lpatiny
lpatiny marked this pull request as draft July 26, 2026 20:04
@lpatiny
lpatiny requested a review from targos July 27, 2026 09:24
@lpatiny
lpatiny marked this pull request as ready for review July 27, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant