Skip to content

Speed up Expr __add__ and __iadd__#1205

Merged
Joao-Dionisio merged 8 commits into
scipopt:masterfrom
Zeroto521:expr/__add__
May 20, 2026
Merged

Speed up Expr __add__ and __iadd__#1205
Joao-Dionisio merged 8 commits into
scipopt:masterfrom
Zeroto521:expr/__add__

Conversation

@Zeroto521
Copy link
Copy Markdown
Contributor

This pr is 1.75x faster than the master branch.

  • Optimized before: 9.0072 seconds
  • Optimized after: 5.1427 seconds
from timeit import timeit

from pyscipopt import Model


m = Model()

n = 1_000
x = m.addMatrixVar(n)
e1 = x.sum()

y = m.addMatrixVar(n)
e2 = y.sum()

number = 100_000
cost = timeit(lambda: e1 + e2, number=number)
print(f"Cost of adding two expressions over {number:,} runs: {cost:.4f} seconds")

Refactor Expr.__add__ and __iadd__ to centralize term merging in a new C helper _to_dict. _to_dict iterates other.terms with PyDict_Next, skips zero coefficients, and either copies or mutates the target dict depending on the copy flag. __add__ now handles numeric additions first and uses _to_dict(copy=True) for Expr+Expr; __iadd__ uses _to_dict(copy=False) to perform in-place merges and returns self. GenExpr and numpy-array cases are preserved. Error messages were slightly adjusted and numeric values are cast to double for correctness and performance.
Add unit tests to verify Expr + Expr and Expr += Expr behavior. test_Expr_add_Expr constructs -x+1 and y-1, checks their string representations and the combined result (including a 0.0 constant term). test_Expr_iadd_Expr verifies in-place addition mutates the left expression, preserves the right expression, and checks their string representations.
Copilot AI review requested due to automatic review settings April 8, 2026 14:53
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes Expr.__add__ / Expr.__iadd__ by moving term merging into a C-level helper (_to_dict) that iterates dictionaries via PyDict_Next, aiming to reduce Python-level overhead in expression addition.

Changes:

  • Reworked Expr.__add__ and Expr.__iadd__ to use a new C-level merge helper for Expr + Expr / Expr += Expr.
  • Added unit tests covering Expr + Expr and Expr += Expr basic behavior.
  • Documented the optimization in the changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/pyscipopt/expr.pxi Implements faster term merging for addition/in-place addition via _to_dict and updates the dunder methods to use it.
tests/test_expr.py Adds tests validating string representations after Expr + Expr and Expr += Expr.
CHANGELOG.md Notes the new optimization in the Unreleased section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/pyscipopt/expr.pxi Outdated
Comment thread src/pyscipopt/expr.pxi Outdated
Comment thread src/pyscipopt/expr.pxi Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 8, 2026

Codecov Report

❌ Patch coverage is 95.65217% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 57.50%. Comparing base (75ccba9) to head (80aa0f1).
⚠️ Report is 13 commits behind head on master.

Files with missing lines Patch % Lines
src/pyscipopt/expr.pxi 95.65% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1205      +/-   ##
==========================================
+ Coverage   57.17%   57.50%   +0.33%     
==========================================
  Files          26       26              
  Lines        5674     5728      +54     
==========================================
+ Hits         3244     3294      +50     
- Misses       2430     2434       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@Joao-Dionisio
Copy link
Copy Markdown
Member

Hey @Zeroto521 , I'll take a look soon. In the meantime, how would you feel about taking a look at qpsolvers/qpsolvers#347? It's about adding SCIP support to an unified QP-solver. I thought about you because you're both into numpy and the matrix API, so it might be a good fit.

By the way, absolutely 0.0000000% pressure to do anything at all, of course!!

@Zeroto521
Copy link
Copy Markdown
Contributor Author

Hey @Zeroto521 , I'll take a look soon. In the meantime, how would you feel about taking a look at qpsolvers/qpsolvers#347? It's about adding SCIP support to an unified QP-solver. I thought about you because you're both into numpy and the matrix API, so it might be a good fit.

By the way, absolutely 0.0000000% pressure to do anything at all, of course!!

Okay, I'll check that out later.

@Joao-Dionisio
Copy link
Copy Markdown
Member

Hey @Zeroto521 ! Just a couple things. With

if (other_v := <double>(<object>v_ptr)) == 0:
      continue

you are silently dropping zero-coefficient terms from other, right? The old code did terms[v] = terms.get(v, 0.0) + c unconditionally, so if other.terms had a key v with coefficient 0.0 that wasn't already in self.terms, the resulting dict would still contain v: 0.0. With this PR it doesn't. Was this taken into account?

And the other thing is about the changelog, can you move it to the unreleased section, please?

Zeroto521 added 2 commits May 20, 2026 21:59
Relocate the "Speed up `Expr.__add__` and `Expr.__iadd__` via the C-level API" bullet from the 6.2.1 section into the top-level "Changed" section of the changelog to remove duplication and better reflect its categorization.
@Zeroto521
Copy link
Copy Markdown
Contributor Author

Hey @Zeroto521 ! Just a couple things. With

if (other_v := <double>(<object>v_ptr)) == 0:
      continue

you are silently dropping zero-coefficient terms from other, right? The old code did terms[v] = terms.get(v, 0.0) + c unconditionally, so if other.terms had a key v with coefficient 0.0 that wasn't already in self.terms, the resulting dict would still contain v: 0.0. With this PR it doesn't. Was this taken into account?

And the other thing is about the changelog, can you move it to the unreleased section, please?

Done

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@Joao-Dionisio Joao-Dionisio self-requested a review May 20, 2026 18:16
Copy link
Copy Markdown
Member

@Joao-Dionisio Joao-Dionisio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Zeroto521 once again!

@Joao-Dionisio Joao-Dionisio merged commit aa0e243 into scipopt:master May 20, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants