Skip to content

fix: heap Max() and PopLast() return wrong element#20

Merged
ammario merged 1 commit into
coder:mainfrom
suykerbuyk:fix/heap-max
Jun 22, 2026
Merged

fix: heap Max() and PopLast() return wrong element#20
ammario merged 1 commit into
coder:mainfrom
suykerbuyk:fix/heap-max

Conversation

@suykerbuyk

Copy link
Copy Markdown
Contributor

Summary

  • Max() returned data[len-1], which is not the maximum in a binary min-heap
  • PopLast() called Remove(Len()-1), removing an arbitrary element instead of the worst
  • This caused incorrect evictions during neighbor selection, silently degrading graph quality

Fix: scan leaf nodes (indices n/2..n-1) to find the true maximum.

Test plan

  • Added TestHeap_MaxAndPopLast — pushes known values, asserts Max/PopLast return the largest
  • Updated TestGraph_AddSearch — expected results now reflect correct eviction behavior
  • go test ./... and go vet ./... pass

In a binary min-heap, the last array element is not necessarily the
maximum. Max() must scan the leaf nodes (indices n/2..n-1) to find
the true maximum. PopLast() used Remove(Len()-1) which removed an
arbitrary element instead of the worst.

This caused incorrect evictions during neighbor selection and search
result trimming, degrading graph quality.
@suykerbuyk

Copy link
Copy Markdown
Contributor Author

Hi @ammario — friendly ping on this one. This is the root cause of the near-zero recall I hit with the library: in a min-heap Max()/PopLast() returning data[len-1] means searchLayer evicts the wrong candidate, so good neighbors get dropped during search. This PR plus #21 (replenish hardcoding CosineDistance) and #22 (result set bounded by k instead of efSearch) together take recall from ~0–2/10 back to expected levels. All three are small, independent, and cleanly mergeable. Thanks for merging #19 — would appreciate a look at these when you have a moment.

@ammario ammario left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct fix: in a min-heap the max is a leaf node, not data[len-1]. Verified logic and tests pass.

@ammario ammario merged commit c8a3b11 into coder:main Jun 22, 2026
1 check passed
ammario added a commit to suykerbuyk/hnsw that referenced this pull request Jun 22, 2026
Heap fix (coder#20) and replenish distance fix (coder#21) are already present in
this branch; they landed on main via rebase, so absorb main with the
'ours' strategy to clear the duplicate-commit conflict. Tree is unchanged.
ammario pushed a commit that referenced this pull request Jun 22, 2026
Rewrites layerNode.search() to follow HNSW Algorithm 5 (SEARCH-LAYER):
the result set is bounded by efSearch during exploration, termination
uses the worst-of-results condition, and only the k closest are returned
(with deterministic key tiebreaking).

This fixes severely degraded recall: on a 2000-point, 16-dim Euclidean
benchmark, recall@1 improves from 0.20 to 1.00 and recall@10 from 0.175
to 0.998.

Also clamps efSearch to max(k, efSearch) so Search returns up to k
results even when k exceeds the configured EfSearch.

Builds on #20 (heap Max/PopLast fix) and #21 (replenish distance fix).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants