Add active vision - #1072
Conversation
Adds an ACTIVE_VISION head mode that samples candidate head trajectories every cycle and scores them against the ball, team ball, robot detections and a decaying field coverage map, instead of replaying a fixed pattern. Extracts the existing head mover into a tested library first; behaviour of the other head modes is unchanged. Missing inputs are reported and the update is dropped rather than substituted for. Also fixes a division by zero producing NaN keyframes for a single scan line, and an off centre coverage grid when the field extent is not a multiple of the cell size.
No invented defaults for missing input, throttled logging that names what is missing, drop the update rather than the process, and keep retrying inputs that legitimately arrive late.
Removing build/, install/ or log/ by hand breaks every package that is not rebuilt with it, including ones outside the selection being worked on, and recovering needs a full workspace rebuild.
Signed-off-by: Florian Vahl <florian@flova.de>
Signed-off-by: Florian Vahl <florian@flova.de>
Signed-off-by: Florian Vahl <florian@flova.de>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NcdgjqxnvjxCH8iqfgmTUx
| root_link: | ||
| type: string | ||
| default_value: "base_link" | ||
| description: "Link the sampled camera poses are resolved against. The map to this link transform places the robot on the field." |
There was a problem hiding this comment.
The map to this link transform places the robot on the field. This sentence is grammatically wrong or just really hard to parse.
| control_period: | ||
| type: double | ||
| default_value: 0.05 | ||
| description: "Time step (seconds) the controller advances the setpoint by each cycle. Should match the control loop period." |
| debug: | ||
| enabled: | ||
| type: bool | ||
| default_value: true |
jaagut
left a comment
There was a problem hiding this comment.
Written by GPT-6 on my behalf.
The sampling approach and separation of the planner from ROS integration make sense. Before merging, I’d like to address the following integration issues.
-
Make field-dimension retrieval asynchronous.
The retry timer callsget_parameters_from_other_node(), which internally usesspin_until_future_complete()on the head-mover node. After startup, that node already belongs to the EventsExecutor, so the retry throws when it attempts to attach it to another executor. Starting the parameter blackboard late therefore leaves active vision disabled. The separate timer also runs on the same executor thread, so its synchronous wait still blocks head updates.
Please use an asynchronous request with a response callback and bounded retries. Add a regression test where the parameter blackboard becomes available after the initial attempt. Relevant code -
Synchronize parameter access across the two executor threads.
behave()replacesparams_while the explicitly created detection thread readsparams_.active_vision.map_frame. The listener’s internal mutex does not protect this separate copy. This can race during runtime parameter updates, even when only a scoring weight changes.
Please give detection callbacks a synchronized configuration snapshot and release the lock before blocking TF lookups. The default EventsExecutor serializes its own callbacks, but it does not serialize them with the added detection executor. Relevant code -
Give the timer and controller one source for their period.
The timer uses a hardcoded50ms, whilecontroller.control_periodis independently configurable. Changing that parameter changes the setpoint step without changing the update frequency.
Please derive the controller period from the duration used to create the timer, removing the separate parameter unless changing both together is supported. Relevant code -
Make the exposed parameters match what is actually applied.
sampling.random_seedis never passed to the sampler. Coveragemargin,cell_size, andhalf_lifeare loaded during field-dimension retrieval but ignored by subsequent parameter refreshes.
Please wire the seed through and either implement updates for the coverage settings or declare them read-only. Frame settings should likewise be startup-only unless changing them also rebuilds or invalidates the dependent state. Add focused coverage for the supported update behavior. Relevant code -
Retry search initialization after incomplete joint states.
prev_head_mode_is updated before trajectory construction succeeds. If the current joint-state message lacks a head joint, construction fails and later valid messages never trigger another attempt for that mode.
Please retry initialization once usable joint data arrives. Test an incomplete message followed by a complete message without another mode change. Relevant code
There are also a few optional simplifications:
- Remove
active_vision_hold_position_; it is written and reset but never read. - Record coverage before constructing a fully configured scorer, so preparation happens once per cycle.
- Have debug markers use kinematics directly instead of constructing a scorer and accessing the world model.
- Move the unrelated
AGENTS.mdchanges out of this PR and update comments that still describe trajectory sampling.
The existing unit tests are useful, but they do not exercise the node-level startup and concurrency paths above.
Summary
Adds a sampling based active vision algorithm
Proposed changes
Samples head targets and scores them based on some cost function including ball, field, robot coverage.
Related issues
Superceded #856
Checklist
pixi run buildWe tested this on the IFA 2026