Skip to content

CAMEL-24566: Auto-discover GenAI dependencies in Camel CLI - #26055

Open
atiaomar1978-hub wants to merge 4 commits into
apache:mainfrom
atiaomar1978-hub:feature/CAMEL-24566-genai-dependency-auto-discovery
Open

CAMEL-24566: Auto-discover GenAI dependencies in Camel CLI#26055
atiaomar1978-hub wants to merge 4 commits into
apache:mainfrom
atiaomar1978-hub:feature/CAMEL-24566-genai-dependency-auto-discovery

Conversation

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Summary

Auto-generated by Cursor Agent on behalf of atiaomar1978-hub

Implements CAMEL-24566: Camel JBang auto-discovers GenAI-related dependencies when running or exporting routes.

Changes

  • GenAiDependencyDiscovery — scans YAML, XML, and Java route sources for GenAI component URIs and LangChain4j provider packages
  • Run.java — adds discovered dependencies before launch
  • ExportBaseCommand.resolveDependencies() — same discovery for export, dependency list, and dependency update --scan-routes
  • camel-main-known-dependencies.properties — package-level LangChain4j provider class-to-JAR mappings for runtime download

Behavior

Trigger Dependency added
Route uses GenAI scheme (langchain4j-chat:, openai:, etc.) camel:<scheme>
Route/Java references LangChain4j provider package mvn:dev.langchain4j:langchain4j-ollama:… etc.
GenAI route or provider + --observe or camel.aiObservability.enabled=true camel:ai-observability (catalog ≥ 4.23)

camel.aiObservability.enabled=false opts out. Observability is not added by default without --observe or an explicit property.

Tests

  • GenAiDependencyDiscoveryTest — 15 unit tests including false-positive guards and opt-in observability
  • ExportTest.shouldExportGenAiRouteWithObservability — export integration with --observe

Related

Add GenAiDependencyDiscovery to scan route URIs and LangChain4j provider
classes for camel-jbang run, export, dependency list and dependency update.

- Add camel:ai-observability when GenAI routes are detected (unless disabled)
- Add LangChain4j provider JAR mappings to known-dependencies.properties
- Integrate discovery into Run and ExportBaseCommand.resolveDependencies

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
- Make ai-observability opt-in via --observe or camel.aiObservability.enabled=true
- Load LangChain4j provider mappings from known-dependencies (package keys)
- Scope URI extraction by file type and ignore YAML comments/false positives
- Fix settings file scanning for export dependency resolution

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
@cursor
cursor Bot force-pushed the feature/CAMEL-24566-genai-dependency-auto-discovery branch from aedcb7f to 023e9c6 Compare September 2, 2026 23:05
@atiaomar1978-hub
atiaomar1978-hub marked this pull request as ready for review September 2, 2026 23:56

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No we should not scan the code, we need to find another way that is more similar to what we already do

…ncy resolution

Remove GenAiDependencyDiscovery regex scanning of YAML/XML/Java routes.
GenAI components and LangChain4j providers are resolved through the
existing silent-run download pipeline (DependencyDownloaderComponentResolver
and KnownDependenciesResolver), matching OpenTelemetry and LRA handling.

- Add GenAiDependencyHelper to conditionally add camel:ai-observability
  from settings/profile when GenAI artifacts are already present
- Map GenAiObservabilityImpl in camel-main-known-dependencies.properties
- Replace unit tests with GenAiDependencyHelperTest

Co-authored-by: Cursor <cursoragent@cursor.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

Addressed review feedback in 04413ce — removed route source scanning entirely.

New approach (aligned with existing JBang patterns):

  1. GenAI components — resolved by the existing silent-run pipeline: routes load in stub context → DependencyDownloaderComponentResolver downloads from catalog → dependency= lines captured in run settings (same as any other component).

  2. LangChain4j provider JARs — resolved via KnownDependenciesResolver when bean types / imports are loaded (dev.langchain4j.model.* mappings already in camel-main-known-dependencies.properties).

  3. camel-ai-observability — added by GenAiDependencyHelper using profile/settings rules (like OpenTelemetry/LRA): only when --observe or camel.aiObservability.enabled=true and GenAI artifacts are already in the dependency set from silent run. Also mapped GenAiObservabilityImpl in known-deps for runtime download.

Removed GenAiDependencyDiscovery (315 lines of regex scanning). Tests pass.

Cursor Agent on behalf of atiaomar1978-hub

@atiaomar1978-hub atiaomar1978-hub left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @davsclaus — agreed, route source scanning is removed in 04413ce. GenAI dependencies now use the same silent-run + catalog + known-deps pipeline as other components. Only camel-ai-observability uses a small settings helper (OpenTelemetry/LRA style).

Cursor Agent on behalf of atiaomar1978-hub

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

✅ Generated files are up to date

An earlier CI run reported uncommitted generated changes; the latest run no longer does.

@github-actions github-actions Bot removed the catalog label Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • dsl/camel-jbang/camel-jbang-core
  • dsl/camel-kamelet-main

🔬 Scalpel shadow comparison — Scalpel: 8 tested, 8 compile-only — current: 6 all tested

Maveniverse Scalpel detected 16 affected modules (current approach: 6).

⚠️ Modules only in Scalpel (10)
  • camel-jbang-core
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-kamelet-main
  • camel-launcher
  • coverage

Skip-tests mode would test 8 modules (2 direct + 6 downstream), skip tests for 8 (generated code, meta-modules)

Modules Scalpel would test (8)
  • camel-jbang-core
  • camel-jbang-mcp
  • camel-jbang-plugin-mcp
  • camel-jbang-plugin-route-parser
  • camel-jbang-plugin-tui
  • camel-jbang-plugin-validate
  • camel-kamelet-main
  • camel-launcher-container
Modules with tests skipped (8)
  • camel-jbang-it
  • camel-jbang-main
  • camel-jbang-plugin-edit
  • camel-jbang-plugin-generate
  • camel-jbang-plugin-kubernetes
  • camel-jbang-plugin-test
  • camel-launcher
  • coverage

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

⚠️ Some tests are disabled on GitHub Actions (@DisabledIfSystemProperty(named = "ci.env.name")) and require manual verification:

  • dsl/camel-jbang/camel-jbang-core: 1 test(s) disabled on GitHub Actions

💡 Manual integration tests recommended:

You modified dsl/camel-jbang/camel-jbang-core. The related integration tests in dsl/camel-jbang/camel-jbang-it are excluded from CI. Consider running them manually:

mvn verify -f dsl/camel-jbang/camel-jbang-it -Djbang-it-test
All tested modules (16 modules)
  • Camel :: Coverage
  • Camel :: JBang :: Core
  • Camel :: JBang :: Integration tests
  • Camel :: JBang :: MCP
  • Camel :: JBang :: Main
  • Camel :: JBang :: Plugin :: Edit
  • Camel :: JBang :: Plugin :: Generate
  • Camel :: JBang :: Plugin :: Kubernetes
  • Camel :: JBang :: Plugin :: MCP
  • Camel :: JBang :: Plugin :: Route Parser
  • Camel :: JBang :: Plugin :: TUI
  • Camel :: JBang :: Plugin :: Testing
  • Camel :: JBang :: Plugin :: Validate
  • Camel :: Kamelet Main
  • Camel :: Launcher
  • Camel :: Launcher :: Container

⚙️ View full build and test results

@gnodet gnodet left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current diff (after the 04413ce rework that removed route source scanning). The approach now correctly relies on the existing silent-run + catalog + KnownDependenciesResolver pipeline for GenAI components/providers, with GenAiDependencyHelper only handling the camel-ai-observability opt-in — consistent with how OpenTelemetry and LRA are handled. A few observations below.

This review was generated by an AI agent, Hermès, on behalf of @gnodet.

private static boolean isAiArtifactId(String dep) {
int idx = dep.indexOf(":camel-");
if (idx < 0) {
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isAiArtifactId uses artifact.startsWith("camel-ai-") which matches camel-ai-observability itself. If a user adds mvn:org.apache.camel:camel-ai-observability:... as an explicit --dep, hasGenAiDependency returns true, and the method would add camel:ai-observability again as a duplicate.

In ExportBaseCommand this is harmless (TreeSet dedup), but in Run.java dependencies is an ArrayList, so it results in a duplicate entry. Not a crash-level issue — the downstream dependency resolution likely handles it — but it's an unintended self-reference.

Consider excluding camel-ai-observability explicitly, or using the catalog label check as the primary filter for mvn: deps too.

return artifact.startsWith("camel-langchain4j")
|| artifact.startsWith("camel-openai")
|| artifact.startsWith("camel-spring-ai")
|| artifact.startsWith("camel-aws-bedrock")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contains("-ai-") check is broad. Today's component names are safe, but future non-AI components with -ai- in their artifact ID (e.g., a hypothetical camel-repair-aid-connector) would be false-positived here. The catalog label check (isAiLabel) is the authoritative source of truth — could the mvn: branch also query the catalog by artifact ID instead of relying on string heuristics?

dev.langchain4j.model.vertexai = dev.langchain4j:langchain4j-vertex-ai:${langchain4j-version}
dev.langchain4j.model.googleai = dev.langchain4j:langchain4j-google-ai-gemini:${langchain4j-version}
dev.langchain4j.model.github = dev.langchain4j:langchain4j-github-models:${langchain4j-version}
dev.langchain4j.model.embedding.onnx = dev.langchain4j:langchain4j-embeddings:${langchain4j-beta-version}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last two entries use a different convention from the rest of the file:

org.apache.camel.component.ai.observability.GenAiObservabilityImpl = camel:ai-observability
camel.aiObservability = camel:ai-observability

Existing entries in this file are keyed by either fully-qualified class names or property keys with = escaping (e.g., org.apache.camel.component.activemq.ActiveMQComponent\:embedded\=true). The camel.aiObservability entry is a camel-main property prefix, not a class name — KnownDependenciesResolver.findGav() does prefix-trimming on . separators, so this entry would match any property or class starting with camel.aiObservability. Is this intentional? It means any camel.aiObservability.* property access through the properties component triggers a download of camel:ai-observability, which seems like the desired behavior but is worth calling out since it's a novel use of the known-deps file.

* </p>
*/
public final class GenAiDependencyHelper {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the constant AI_OBSERVABILITY_ENABLED is public but is only used within this class and in tests (which access package-private methods anyway). Consider narrowing to package-private if there's no external consumer planned.

GenAiDependencyHelper.addAiObservabilityIfNeeded(deps, properties, true, catalog);

assertThat(deps).doesNotContain("camel:ai-observability");
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test mocks the catalog to return a ComponentModel with label "ai" for "openai" — but the real catalog already has openai with label "ai" (verified in components/camel-ai/camel-openai/src/generated/resources/). Using the real DefaultCamelCatalog here (like the other tests) would be more resilient to future catalog changes and wouldn't require Mockito.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants