Skip to content

feat: fixture-backed refactor-process eval - #108

Merged
igmarin merged 1 commit into
mainfrom
eval/refactor-process-fixture
Sep 14, 2026
Merged

igmarin merged 1 commit into
mainfrom
eval/refactor-process-fixture

Conversation

@igmarin

@igmarin igmarin commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Why

ROADMAP item 1: fourth extract-style eval. refactor-process/basic named OrderProcessor but shipped no class, so the agent invented one.

What

  • Duplicated process_standard / process_express (same sku/quantity guards, different shipping)
  • Characterization tests already Green
  • Task: extract helpers one step at a time; do not change hashes
  • Condensed skills/refactor-process/SKILL.md
  • Bench test: two copies of each guard, no validate/price helpers yet, tests pass, skill example ruby -cs

Proof

ruby evals/skills/refactor-process/basic/order_processor_test.rb
# 4 runs, 0 failures

bundle exec ruby -Itest test/evals/refactor_process_basic_test.rb
# 4 runs, 0 failures

bundle exec ruby -Itest test/evaluator/source_path_resolver_test.rb
# 12 runs, 0 failures

Ship OrderProcessor with duplicated standard/express paths and passing
characterization tests. The agent extracts helpers without changing
return values. Include a condensed skill and a bench test that the
duplication is still present.
@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 5 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: CHILL

Plan: Advanced

Run ID: cdbdf318-5a2d-4b78-9803-a4e5902eb926

📥 Commits

Reviewing files that changed from the base of the PR and between d709344 and 50ee6ab.

📒 Files selected for processing (7)
  • CHANGELOG.md
  • evals/skills/refactor-process/basic/criteria.json
  • evals/skills/refactor-process/basic/order_processor.rb
  • evals/skills/refactor-process/basic/order_processor_test.rb
  • evals/skills/refactor-process/basic/task.md
  • skills/refactor-process/SKILL.md
  • test/evals/refactor_process_basic_test.rb

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@igmarin
igmarin merged commit 3f46789 into main Sep 14, 2026
6 checks passed
@igmarin
igmarin deleted the eval/refactor-process-fixture branch September 14, 2026 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant