Enable Knowledge Layer and migrate IoTDB to native Node.js client - #119
Enable Knowledge Layer and migrate IoTDB to native Node.js client#119chrizmc wants to merge 9 commits into
Conversation
chrizmc
commented
Jun 17, 2026
chrizmc
commented
- Prepare RDF4J reasoner integration with new adapter
- Restructure symbolic-reasoner into adapters directory (rdfox, rdf4j, pellet)
- Upgrade IoTDB to V 2.0.8
- Implement read/write on new native IoTDB Node.js client
- Implement subscription/polling and unsubscribe on new IoTDB client
- Full cutover to new client and legacy Thrift path removal
- Remove generated Thrift code (IClientRPCService.js, client_types.js, common_types.js)
- Add RDF4JAdapter C++ class implementing IReasonerAdapter interface - Add InferenceEngineType::RDF4J enum with string conversion support - Restructure symbolic-reasoner adapters into dedicated adapters/ directory - Moved rdfox, rdf4j-reasoner, and pellet adapters to symbolic-reasoner/adapters/ - Added MODEL_CONFIGURATION_PATH environment variable support - Fixed getPathToUseCases() to return dynamic value instead of static initialization - Updated CMakeLists.txt paths to reflect new adapter structure - Make model config path configurable via MODEL_CONFIGURATION_FILE env var - Add README documentation for adapter and engine integration - Created docker/rdf4j and docker/rdfox - docs: Add reasoner integration guide - Delete unused files Signed-off-by: Haonan Qiu <haonan.qiu@bmw.de>
* Add missing RDFox model files * Add IoTDB migration controls and client smoke-test foundation Introduce operation-level migration path configuration with safe legacy defaults, startup visibility, and routing hooks to support gradual transition from legacy behavior to the new IoTDB client path. Also add Docker and smoke-test support for validating @iotdb/client import and session/query connectivity during migration setup. * Split HandlerBase into smaller, focused modules HandlerBase.ts was doing too much. Moved formatting, schema loading, validation, and response building into their own utility files. Same behavior, easier to read and maintain. * Refactor IoTDB operation routing through adapters Introduce an adapter layer for IoTDB operations by adding a shared operation contract, a legacy adapter, and a new-path stub adapter. Update HandlerBase/IoTDBHandler wiring so operation dispatch goes through adapters while preserving legacy behavior by default, and adjust IoTDB handler tests to validate the new delegation flow. * Fix eslint config errors * Add missing function docs
…OVESA#94) * Wire new get path behind migration flag - update IoTDBHandler get-path routing integration - update NewIoTDBAdapter get-path behavior for incremental migration - keep legacy fallback behavior unchanged * Route set through new adapter with legacy fallback - add new-path debug marker for set operation - make NewIoTDBAdapter.set callable under migration flag - keep legacy execution path as temporary fallback * Extract shared setCore path for legacy and new adapters - rename IoTDBHandler setLegacy implementation to setCore - keep setLegacy as wrapper to preserve current behavior - extend LegacyIoTDBPort with setCore contract - route NewIoTDBAdapter.set through setCore instead of setLegacy * Implement new-client get path and add handler/session tests - add NewIoTDBSession wrapper using @iotdb/client - implement IoTDBHandler.getNewClient with lazy session open and response forwarding - route NewIoTDBAdapter.get to new-client handler path - close new session on client unsubscribe lifecycle - add dedicated NewIoTDBSession unit tests (happy path, metadata/null handling, errors, lifecycle) - extend IoTDBHandler tests for new get routing and response/error behavior * Implement new-client set path with session write tests - add NewIoTDBSession.setDataPoints using @iotdb/client write API - wire IoTDBHandler.setNewClient to new session write flow - preserve existing success/error response contract - add handler tests for set routing and success/failure behavior - add NewIoTDBSession setDataPoints unit tests (success, lazy open, failure, type validation)" * Improve NewIoTDBSession coverage and extend type mapping - extend NewIoTDBSession data type mapping for int8/uint8/int16/uint16/uint32 - restructure NewIoTDBSession tests with grouped describe blocks (lifecycle/get/set) - add setDataPoints tests for success, lazy open, length mismatch, insert failure, and unsupported type - keep getDataPoints behavior tests for happy path, null handling, malformed metadata, and query failure * setDataPoints falls back to STRING for unsupported data type * Add set data points documentation
…TDB client (COVESA#95) * Add new-path subscribe/unsubscribe routing and handler behavior with tests * Delegate new subscribe/unsubscribe via NewIoTDBSession bridge * Harden new subscribe/unsubscribe behavior and lifecycle cleanup coverage
* Cut over to new client defaults and remove legacy routing scaffolding * Complete new-client cutover and remove legacy thrift path
There was a problem hiding this comment.
Sanity Testing
- SUCCESS: Built reference docker compose from cdsp/docker directory (without using cache) and could start components with exception of RDFox (no license)
- SUCCESS: Sanity tested VISSR could still get data from IoTDB after IoTDB 2.0.8 update using VISSR web client
- SUCCESS: Built docker compose for KL/IL example
- FAILURE: Running docker compose for KL/IL example using RD4J profile failed with runtime error. See detail below.
KL/IL example runtime failure error:
cdsp/examples/knowledgelayer-hello-world$ docker compose --profile rdf4j up
[+] Running 1/1
✘ rdf4j-stream-reasoner-service Error pull access denied for rdf4j-stream-reasoner, repositor... 1.1s
Error response from daemon: pull access denied for rdf4j-stream-reasoner, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
General
Whilst ultimately not a barrier to merge this time the PR has a single commit with a huge bundle of changes that appear to not be so tightly coupled that they couldn't be split. For example three major themes: IoTDB version update, Node.js client integration, RDF4J reasoner.
Huge commits can make it harder to parse project history. Whilst grouping of changes can be a personal preference I think it would be good to discuss have things could be different next time.
There was a problem hiding this comment.
The profile concepts introduced here, whilst welcome e.g the ability to avoid runtime error due to missing RDFox license, have not been carried over into the reference docker-compose in the cdsp\docker directory.
I presume RDF4J support is also missing.
We need to discuss how/when to address this gap
There was a problem hiding this comment.
Support for RDF4J in the reference docker-compose remains missing in the updated PR.
There was a problem hiding this comment.
A related question would be when RDF4J support is in the reference docker compose whether this compose could be simplified by taking services from it. Doing so may simplify maintenance.
|
Hi, review comments aside thank you for the work. Great to see the removal of all the Node.js Thrift bindings in the IL code. Along with the RDF4J work. |
- Comment out all rdf4j service definitions in knowledgelayer-hello-world
docker-compose.yml with a TODO explaining that the Java reasoner service
and C++ adapter are not yet implemented; mark blocks with [RDF4J] for
easy re-enable once the work is completed and merged
- Update hello-world README: replace rdf4j run/stop instructions with a
temporary unavailability note pointing to docker-compose.yml
- Fix broken IoTDB documentation URLs in apache-iotdb.md and
vehicle-speed-downsample-iotdb README (section anchors changed upstream)
Signed-off-by: Haonan Qiu <haonan.qiu@bmw.de>
d527a56 to
c1eed4a
Compare
- add RDF4J adapter and reasoner service support for rules, inference, and datastore deletion - restore and document RDF4J as an alternative reasoning engine in the Knowledge Layer and hello-world example - add RDF4J hello-world model assets and REST API test coverage - fix query, rule, and fixture path issues that caused shared-fixture and adapter integration test failures - align RDF4J cleanup and integration-test behavior with the RDFox adapter pattern Signed-off-by: Haonan Qiu <haonan.qiu@bmw.de>
c1eed4a to
e61be28
Compare
slawr
left a comment
slawr
left a comment
There was a problem hiding this comment.
Hi @claireqiu thanks for the updates. Unfortunately I must reject the PR as the new commits from Juan are missing a signed off in the commit message or a different form of DCO. If signed-off is acceptable please ask Juan to add one to his commit messages and update. Thanks
e61be28 to
0098645
Compare
so I re-commited with my sign-off and authored by Juan. is this fine? |
Yes, that that meets the requirement of a DCO to assert that the contribution is freely given within the terms of the project including the license thanks. |
Signed-off-by: Haonan Qiu <haonan.qiu@bmw.de>
0098645 to
b099c08
Compare
slawr
left a comment
slawr
left a comment
There was a problem hiding this comment.
Sanity Testing
SUCCESS: Built and ran docker compose for KL/IL example using RDF4J docker profile
SUCCESS: Built and ran CDSP reference docker compose (RDFox). Started IoTDB and VISSR. Used VISSR web client to query IoTDB data.
There was a problem hiding this comment.
Is there a reason the RDF4J Service source is located in the Docker source tree, rather than somewhere in the CDSP core source tree in cdsp/cdsp with the other building blocks? Typically the Docker tree is reserved for Docker specific configurations files such as Dockerfiles or Docker Compose service descriptions.
Please discuss with the maintainers.
There was a problem hiding this comment.
It is mentioned down in the KL driving style example sources, but the RDF4J support has not been integrated into this reference docker compose for CDSP. This compose and the README is intended to represent a reference for running the core building blocks (VISSR, IoTDB, KL, IL) of the Playground. Newcomers are pointed here and the examples as a way to get started.
Please discuss with the CDSP maintainers.
There was a problem hiding this comment.
A related question would be when RDF4J support is in the reference docker compose whether this compose could be simplified by taking services from it. Doing so may simplify maintenance.
There was a problem hiding this comment.
The content of this Dockerfile seems heavily similar to the cdsp/knowledge-layer/Dockerfile. Is a modified duplicate really required here? Are there changes here that should be migrated to the core component?
There was a problem hiding this comment.
Observation: The KL driving style example docker compose relies on you always specifying the reasoner profile. For example docker compose down would not shutdown all containers, only the base ones. To shutdown all containers you need to use docker compose --profile rdf4j down. The example CLI commands shown in the README all meet that criteria.
I do not see it as a barrier to merging, but we could collectively discuss how to automate a default in the future. Docker provides mechanisms for this but we should collectively decide which to use.
3 participants
