Skip to content

[Feat] Few-shot 비교 평가 사용량 계측 추가 - #318

Merged
whc9999 merged 1 commit into
devfrom
feat/fewshot-evaluation-usage-metrics
Sep 16, 2026
Merged

whc9999 merged 1 commit into
devfrom
feat/fewshot-evaluation-usage-metrics

Conversation

@whc9999

@whc9999 whc9999 commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

✨ 어떤 이유로 PR를 하셨나요?

  • feature 병합
  • 버그 수정(아래에 issue #를 남겨주세요)
  • 코드 개선
  • 코드 수정
  • 배포
  • 기타(아래에 자세한 내용 기입해주세요)

📋 세부 내용 - 왜 해당 PR이 필요한지 작업 내용을 자세하게 설명해주세요

  • OpenAI 분석 응답의 입력 및 출력 토큰 사용량 수집
  • single-pass와 two-pass 평가 결과에 토큰 사용량 전달
  • 선택적 recheck 호출 토큰을 final 사용량에 합산
  • 평가 CSV의 candidate/final/total 토큰 컬럼 기록
  • Few-shot 선택 sidecar에 케이스별 Cohere 논리 호출 수 기록
  • STATIC/DYNAMIC 20건 사용량 재평가 결과 문서화
  • 토큰 및 Cohere 호출 수 회귀 테스트 추가

📸 작업 화면 스크린샷

⚠️ PR하기 전에 확인해주세요

  • 로컬테스트를 진행하셨나요?
  • 머지할 브랜치를 확인하셨나요?
  • 관련 label을 선택하셨나요?

🚨 관련 이슈 번호 [#293 ]

- OpenAI 분석 응답의 입력 및 출력 토큰 사용량 수집
- single-pass와 two-pass 평가 결과에 토큰 사용량 전달
- 선택적 recheck 호출 토큰을 final 사용량에 합산
- 평가 CSV의 candidate/final/total 토큰 컬럼 기록
- Few-shot 선택 sidecar에 케이스별 Cohere 논리 호출 수 기록
- STATIC/DYNAMIC 20건 사용량 재평가 결과 문서화
- 토큰 및 Cohere 호출 수 회귀 테스트 추가
@whc9999 whc9999 self-assigned this Sep 16, 2026
@whc9999 whc9999 added the ✨ feat New feature or request label Sep 16, 2026
@coderabbitai

coderabbitai Bot commented Sep 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 7cb84426-13dd-46a0-8a2a-e14bd9f8b823

📥 Commits

Reviewing files that changed from the base of the PR and between 657c087 and 8564bb3.

📒 Files selected for processing (16)
  • docs/evaluation/fewshot-static-dynamic-20260916.md
  • docs/fewshot-evaluation-metadata.md
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/infrastructure/ai/OpenAiAnalysisAdapter.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/ai/AnalysisAiClient.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/ai/AnalysisPromptBuilder.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/ai/fewshot/DefaultFewShotSearchService.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/ai/fewshot/FewShotSearchService.java
  • src/main/java/com/jobdri/jobdri_api/domain/analysis/service/ai/fewshot/FewShotSelectionMetadata.java
  • src/main/java/com/jobdri/jobdri_api/domain/evaluation/analysis/EvaluationAnalysisBatchService.java
  • src/main/java/com/jobdri/jobdri_api/domain/evaluation/analysis/model/EvaluationGeneratedResult.java
  • src/main/java/com/jobdri/jobdri_api/domain/evaluation/infrastructure/analysis/AnalysisAiEvaluationAnalysisGenerator.java
  • src/main/java/com/jobdri/jobdri_api/global/cohere/CohereEmbeddingClient.java
  • src/test/java/com/jobdri/jobdri_api/domain/analysis/infrastructure/ai/OpenAiAnalysisAdapterTest.java
  • src/test/java/com/jobdri/jobdri_api/domain/analysis/service/ai/FewShotMetadataPromptTest.java
  • src/test/java/com/jobdri/jobdri_api/domain/evaluation/analysis/EvaluationAnalysisBatchServiceTest.java
  • src/test/java/com/jobdri/jobdri_api/global/cohere/CohereEmbeddingClientTest.java

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@whc9999
whc9999 merged commit 4568a2a into dev Sep 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

✨ feat New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant