Skip to content

CAMEL-24559: Extend GenAI observability to OpenAI embeddings, moderation, and responses - #26106

Merged
davsclaus merged 3 commits into
apache:mainfrom
atiaomar1978-hub:cursor/CAMEL-24559-openai-genai-observability-c587
Sep 5, 2026
Merged

CAMEL-24559: Extend GenAI observability to OpenAI embeddings, moderation, and responses#26106
davsclaus merged 3 commits into
apache:mainfrom
atiaomar1978-hub:cursor/CAMEL-24559-openai-genai-observability-c587

Conversation

@atiaomar1978-hub

@atiaomar1978-hub atiaomar1978-hub commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to CAMEL-23861 / CAMEL-24559: extend GenAI observability to additional OpenAI producer operations beyond chat-completion.

This PR instruments OpenAI embeddings, moderation, and responses with the existing GenAiObservability API.

Operation gen_ai.operation.name
openai:embeddings embeddings
openai:moderation moderation (Camel extension)
openai:responses chat
openai:chat-completion chat (unchanged)

Changes

  • Instrument OpenAIEmbeddingsProducer, OpenAIModerationProducer, OpenAIResponsesProducer
  • Add GenAiOperationName.MODERATION
  • Tests: OpenAIEmbeddingsObservabilityTest, OpenAIModerationObservabilityTest, OpenAIResponsesObservabilityTest
  • Docs: ai-observability.adoc, catalog mirror, 4.23 upgrade guide

Test plan

./mvnw -pl components/camel-ai/camel-openai -am test -Dtest=OpenAIEmbeddingsObservabilityTest,OpenAIModerationObservabilityTest,OpenAIResponsesObservabilityTest

Images, audio, Spring AI modules, and cloud LLMs remain for future CAMEL-24559 follow-ups.

AI-generated PR description on behalf of @atiaomar1978-hub

@atiaomar1978-hub

atiaomar1978-hub commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Links CAMEL-24559 — first follow-up slice for extending GenAI observability beyond chat/streaming.

Review status: Bugbot (no bugs) and Grok review feedback addressed in commit 38b10e2fc6c (SDK-only span scope, chat for responses, docs, stronger tests).

AI-generated comment on behalf of @atiaomar1978-hub

@atiaomar1978-hub atiaomar1978-hub mentioned this pull request Sep 4, 2026
Closed
1 task
@atiaomar1978-hub

atiaomar1978-hub commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Links CAMEL-24559 — first follow-up slice. Bugbot + Grok review feedback addressed in 38b10e2.

@atiaomar1978-hub
atiaomar1978-hub marked this pull request as ready for review September 4, 2026 03:40
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions

github-actions Bot commented Sep 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • catalog/camel-catalog
  • components/camel-ai/camel-ai-observability-api
  • components/camel-ai/camel-ai-observability
  • components/camel-ai/camel-openai
  • docs

ℹ️ Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the test-dependents label to force testing all dependents.

🔬 Scalpel shadow comparison — Scalpel: 73 tested, 25 compile-only — current: 71 all tested

Maveniverse Scalpel detected 98 affected modules (current approach: 71).

⚠️ Modules only in Scalpel (27)
  • apache-camel
  • camel-allcomponents
  • camel-catalog
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • docs
  • dummy-component

Skip-tests mode would test 73 modules (5 direct + 68 downstream), skip tests for 25 (generated code, meta-modules)

Modules Scalpel would test (73)
  • camel-a2a
  • camel-ai-observability
  • camel-ai-observability-api
  • camel-as2
  • camel-catalog
  • camel-clickup
  • camel-consul
  • camel-core-all
  • camel-csv
  • camel-fhir
  • camel-groovy
  • camel-hazelcast
  • camel-hl7
  • camel-ibm-watsonx-ai
  • camel-iso8583
  • camel-java-joor-dsl
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-jetty
  • camel-jsonpath
  • camel-kamelet
  • camel-kamelet-main-support
  • camel-knative-http
  • camel-langchain4j-agent
  • camel-langchain4j-chat
  • camel-langchain4j-embeddings
  • camel-langchain4j-embeddingstore
  • camel-launcher-container
  • camel-mail
  • camel-mail-microsoft-oauth
  • camel-main
  • camel-maven-plugin
  • camel-mcp-server
  • camel-micrometer-prometheus
  • camel-microprofile-health
  • camel-netty
  • camel-netty-http
  • camel-oauth
  • camel-observability-services
  • camel-once
  • camel-openai
  • camel-openapi-validator
  • camel-platform-http-main
  • camel-platform-http-vertx
  • camel-plc4x
  • camel-pubnub
  • camel-quickfix
  • camel-rest-openapi
  • camel-restdsl-openapi-plugin
  • camel-salesforce
  • camel-salesforce-codegen
  • camel-salesforce-maven-plugin
  • camel-spring-ai-chat
  • camel-spring-main
  • camel-state-store
  • camel-syslog
  • camel-telegram
  • camel-test-main-junit5
  • camel-test-main-junit6
  • camel-vertx-websocket
  • camel-webhook
  • camel-whatsapp
  • camel-xml-io-dsl
  • camel-xml-jaxb-dsl
  • camel-xml-jaxb-dsl-test-definition
  • camel-xml-jaxb-dsl-test-spring
  • camel-yaml-dsl-validator
  • camel-yaml-dsl-validator-maven-plugin
  • camel-zookeeper
  • docs
Modules with tests skipped (25)
  • apache-camel
  • camel-allcomponents
  • camel-catalog-console
  • camel-catalog-maven
  • camel-catalog-suggest
  • camel-componentdsl
  • camel-endpointdsl
  • camel-endpointdsl-support
  • camel-itest
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • camel-report-maven-plugin
  • camel-route-parser
  • camel-yaml-dsl
  • camel-yaml-dsl-deserializers
  • camel-yaml-dsl-maven-plugin
  • coverage
  • dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • components/camel-ai/camel-openai: 7 test(s) disabled on GitHub Actions
Build reactor — dependencies compiled but only changed modules were tested (5 modules)
  • Camel :: AI :: Observability
  • Camel :: AI :: Observability API
  • Camel :: AI :: OpenAI
  • Camel :: Catalog :: Camel Catalog
  • Camel :: Docs

⚙️ View full build and test results

gnodet
gnodet reviewed Sep 4, 2026
View reviewed changes

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary: Solid and well-structured extension of GenAI observability to three additional OpenAI producers. The pattern follows the established createChatCompletion in OpenAIProducer closely. Two issues worth addressing before merge.

This review was generated by an automated reviewer (Hermès) on behalf of @gnodet.

Comment thread ...i/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIEmbeddingsProducer.java Outdated
Comment thread ...ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java Outdated
Comment thread ...ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java Outdated
Comment thread ...i/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIModerationProducer.java
cursoragent and others added 3 commits September 4, 2026 22:02
@cursoragent
CAMEL-24559: Add GenAI observability for OpenAI embeddings, moderatio…
c6764d7 
…n, and responses

Extend camel-openai producers beyond chat-completion with OpenTelemetry
spans and Micrometer metrics using the existing GenAiObservability API.

- Instrument OpenAIEmbeddingsProducer, OpenAIModerationProducer, and
  OpenAIResponsesProducer with try/recordSuccess/recordError/close
- Add GenAiOperationName.MODERATION for content-policy operations
- Add OpenAIEmbeddingsObservabilityTest, OpenAIModerationObservabilityTest,
  OpenAIResponsesObservabilityTest, and shared test support
- Document OpenAI operation coverage in ai-observability.adoc and 4.23
  upgrade guide

Co-authored-by: Cursor Agent <noreply@cursor.com>
@cursoragent
CAMEL-24559: Address review feedback on OpenAI observability
bcdb8f7 
- Scope GenAI spans to SDK calls only (embeddings/moderation)
- Map openai:responses to gen_ai.operation.name=chat per OTel OpenAI usage
- Clarify MODERATION as Camel extension in enum javadoc and docs
- Strengthen tests: body/header assertions, exact token tags, override properties

Co-authored-by: Cursor Agent <noreply@cursor.com>
@cursoragent
CAMEL-24559: Address gnodet review feedback on OpenAI observability
d1a3fc2 
- Pass OpenAI token counts as Long without Math.toIntExact (includes
  CAMEL-24560 GenAiUsage Long token fields cherry-picked from follow-up)
- Extract observedCall helper in OpenAIResponsesProducer to deduplicate
  observation boilerplate between createResponse and createStructuredResponse
- Restore moderation security comments explaining why mismatched result
  counts must fail the exchange

Co-authored-by: Cursor Agent <noreply@cursor.com>
@cursor
cursor Bot force-pushed the cursor/CAMEL-24559-openai-genai-observability-c587 branch from 067e3b1 to d1a3fc2 Compare September 4, 2026 22:03
@atiaomar1978-hub

atiaomar1978-hub commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

This comment was generated by an AI agent on behalf of @atiaomar1978-hub.

Review feedback from @gnodet has been addressed and the branch rebased onto latest main:

  1. Long token counts — Removed Math.toIntExact / toTokenCount from OpenAIEmbeddingsProducer and OpenAIResponsesProducer; token counts are passed directly as long values (CAMEL-24560 is already on main).

  2. OpenAIResponsesProducer deduplication — Extracted shared observedCall(exchange, model, ThrowingSupplier) helper used by both createResponse and createStructuredResponse.

  3. Moderation security comments — Restored the comments explaining why a mismatched moderation result count must fail the exchange (content-gating invariant) and why the full response is stored only after validation.

Rebased commits on main:

  • c6764d78 — initial observability instrumentation
  • bcdb8f7f — first review round (span scope, tests, docs)
  • d1a3fc28 — gnodet review fixes

Tests run locally after rebase: GenAiUsageTest, OpenAIEmbeddingsObservabilityTest, OpenAIModerationObservabilityTest, OpenAIResponsesObservabilityTest — all green.

@davsclaus davsclaus added this to the 4.23.0 milestone Sep 5, 2026
@davsclaus davsclaus added the enhancement New feature or request label Sep 5, 2026
@davsclaus
davsclaus merged commit 220e232 into apache:main Sep 5, 2026
6 checks passed
gnodet
gnodet approved these changes Sep 5, 2026
View reviewed changes

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after d1a3fc2: All four findings from the previous review are addressed.

  1. Math.toIntExact removed (Embeddings) — usage.promptTokens() passed directly to GenAiUsage.of(Long, ...), matching the OpenAIProducer.createChatCompletion pattern.
  2. Math.toIntExact removed (Responses) — usage.inputTokens() / usage.outputTokens() passed directly as long.
  3. observedCall helper extracted (Responses) — Shared helper eliminates the observation boilerplate duplication between createResponse and createStructuredResponse. Clean implementation using ThrowingSupplier.
  4. Security comment restored (Moderation) — The content-gating invariant comment is present.

No new issues. The observation patterns are consistent with OpenAIProducer.createChatCompletion, tests cover all three new producers, and the docs are updated.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gnodet gnodet gnodet approved these changes
@davsclaus davsclaus davsclaus approved these changes

Assignees

@atiaomar1978-hub atiaomar1978-hub

Labels

catalog components components-ai docs enhancement New feature or request

Projects

None yet

Milestone

4.23.0

Development

Successfully merging this pull request may close these issues.

4 participants

@atiaomar1978-hub @gnodet @davsclaus @cursoragent