Retain UI render world data.#24893
Conversation
At present, Bevy re-extracts all render world data for the UI from scratch every frame. This is wasteful, as most UI widgets don't change from frame to frame. This PR changes Bevy UI to retain phase items and render-world entities from frame to frame in the render world. It uses ECS change detection to only re-extract UI widgets that have changed in some way. UI widgets differ from 2D and 3D meshes in that a single UI widget in the main world may extract to multiple phase items in the render world. For UI nodes, multiple systems process the same node repeatedly in order to generate the various rendered pieces (the base node, drop shadows, backgrounds, etc.) To handle this, I split out the change detection and invalidation logic into a separate system, `extract_uinode_changes`, that runs before any of the main extraction systems. When that system detects a change to any of the components that are relevant to rendering, all render-world entities corresponding to the changed main-world entity are despawned and created anew. This isn't as efficient as it could be, but it's fine for now and is a sizable improvement to the current situation. Note that one unfortunate side effect of these changes is that we now mark UI nodes as changed if their visuals change in any way, as opposed to only when their layout changes. The layout code uses the `Changed` flag to determine whether reflow needs to be performed, so reflows will now be needlessly performed if e.g. the color of a node changes, even though color only affects visuals and not layout and so the reflow is unnecessary. I believe that this use of the changed flag by layout was always an abuse, because `Changed` is supposed to be set whenever a component changes, not just when some part of a component relevant to some arbitrary system (in this case, layout) changes. In other words, the `Changed` flag should be for every system to use; layout shouldn't have exclusive use of it. If we care about making these types of visual-only changes fast again, layout should be fixed to maintain its own dirty flags in a follow-up. Several potential improvements have been delegated to follow-ups. I opted to perform these as follow-ups to this PR, as this patch is already quite large. 1. The phase items in the `SortedRenderPhase` should be retained as well. This will result in a large speedup, as `extract_ui_camera_view` and `queue_uinodes` time will drop to zero if no changes were made from the previous frame, and `sort_phase_system` will likely improve as well as the data will be in mostly-sorted order. (See attached profile.) 2. The `UiBatch` component is redundant, as the phase items and `ExtractedUiNodes` already contain all the relevant data. I opted to leave it for now and clean it up later. 3. A large chunk of the GPU data that `prepare_uinodes` and similar (`prepare_shadows`, etc.) emit is redundant, as it's duplicated for every vertex. This should be moved to instance-rate shading (and, in fact, as every UI phase item represents a quad, the entire UI subsystem can be moved to *only* emit per-instance data). 4. `prepare_uinodes` and similar duplicate in an ad-hoc way what `batch_and_prepare_sorted_render_phase` does. These systems should be removed in favor of the generic functionality. 5. Much of the work that `prepare_uinodes` and similar do should be moved to the extraction phase so that it can be retained from frame to frame. 6. UI should support GPU preprocessing to eliminate the work that remains. As mentioned before, this work should be a series of follow-ups. On `many_buttons`, with this patch I observed the median frame time to decrease from 48.29 ms to 45.16 ms, a 1.07× speedup. Note that the true value of this patch as far as performance is concerned isn't what the patch itself does but rather what it unlocks in the future.
|
It looks like your PR has been selected for a highlight in the next release blog post, but you didn't provide a release note. Please review the instructions for writing release notes, then expand or revise the content in the release notes directory to showcase your changes. |
ickshonpe
left a comment
There was a problem hiding this comment.
Minimizing any of the examples results in a panic:
cargo run --example text --features="debug"
thread 'Compute Task Pool (2)' (29568) panicked at crates\bevy_anti_alias\src\taa\mod.rs:337:14:
Camera entity wasn't synced.: InvalidEntityError { entity: 320v0, current_generation: EntityGeneration(1) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'Compute Task Pool (2)' (29568) panicked at crates\bevy_ecs\src\error\handler.rs:141:1:
Encountered an error in system `bevy_anti_alias::taa::extract_taa_settings`: System panicked
Encountered a panic in system `bevy_anti_alias::taa::extract_taa_settings`!
thread '<unnamed>' (34620) panicked at crates\bevy_ui_render\src\lib.rs:1175:14:
Camera entity wasn't synced.: InvalidEntityError { entity: 320v0, current_generation: EntityGeneration(1) }
thread '<unnamed>' (34620) panicked at crates\bevy_ecs\src\error\handler.rs:141:1:
Encountered an error in system `bevy_ui_render::extract_ui_camera_view`: System panicked
Encountered a panic in system `bevy_ui_render::extract_ui_camera_view`!
2026-07-06T19:53:27.392979Z WARN bevy_ecs::world::command_queue: CommandQueue has un-applied commands being dropped. Did you forget to call SystemState::apply?
2026-07-06T19:53:27.393097Z WARN bevy_ecs::world::command_queue: CommandQueue has un-applied commands being dropped. Did you forget to call SystemState::apply?
2026-07-06T19:53:27.393198Z WARN bevy_ecs::world::command_queue: CommandQueue has un-applied commands being dropped. Did you forget to call SystemState::apply?
| cached_ui_view_data.insert( | ||
| main_entity, | ||
| CachedUiViewData { | ||
| extracted_view_entity: entity_commands.id(), |
There was a problem hiding this comment.
Looks like the crash is because it stores the wrong view entity:
| extracted_view_entity: entity_commands.id(), | |
| extracted_view_entity: ui_camera_view, |
| let retained_view_entity = | ||
| RetainedViewEntity::new(main_entity.into(), None, UI_CAMERA_SUBVIEW); | ||
| // Creates the UI view. | ||
| let ui_camera_view = commands |
There was a problem hiding this comment.
There is a leak here I think. A new view entity is spawned on changes to the camera target. But old view entities are only despawned if the camera is removed or not renderable, so everytime you resize the window it leaves behind an orphaned view entity.
There was a problem hiding this comment.
Should be fixed. I tested resizing the window and it doesn’t leak now.
At present, Bevy re-extracts all render world data for the UI from scratch every frame. This is wasteful, as most UI widgets don't change from frame to frame.
This PR changes Bevy UI to retain phase items and render-world entities from frame to frame in the render world. It uses ECS change detection to only re-extract UI widgets that have changed in some way.
UI widgets differ from 2D and 3D meshes in that a single UI widget in the main world may extract to multiple phase items in the render world. For UI nodes, multiple systems process the same node repeatedly in order to generate the various rendered pieces (the base node, drop shadows, backgrounds, etc.) To handle this, I split out the change detection and invalidation logic into a separate system,
extract_uinode_changes, that runs before any of the main extraction systems. When that system detects a change to any of the components that are relevant to rendering, all render-world entities corresponding to the changed main-world entity are despawned and created anew. This isn't as efficient as it could be, but it's fine for now and is a sizable improvement to the current situation.Note that one unfortunate side effect of these changes is that we now mark UI nodes as changed if their visuals change in any way, as opposed to only when their layout changes. The layout code uses the
Changedflag to determine whether reflow needs to be performed, so reflows will now be needlessly performed if e.g. the color of a node changes, even though color only affects visuals and not layout and so the reflow is unnecessary. I believe that this use of the changed flag by layout was always an abuse, becauseChangedis supposed to be set whenever a component changes, not just when some part of a component relevant to some arbitrary system (in this case, layout) changes. In other words, theChangedflag should be for every system to use; layout shouldn't have exclusive use of it. If we care about making these types of visual-only changes fast again, layout should be fixed to maintain its own dirty flags in a follow-up.Several potential improvements have been delegated to follow-ups. I opted to perform these as follow-ups to this PR, as this patch is already quite large.
The phase items in the
SortedRenderPhaseshould be retained as well. This will result in a large speedup, asextract_ui_camera_viewandqueue_uinodestime will drop to zero if no changes were made from the previous frame, andsort_phase_systemwill likely improve as well as the data will be in mostly-sorted order. (See attached profile.)The
UiBatchcomponent is redundant, as the phase items andExtractedUiNodesalready contain all the relevant data. I opted to leave it for now and clean it up later.A large chunk of the GPU data that
prepare_uinodesand similar (prepare_shadows, etc.) emit is redundant, as it's duplicated for every vertex. This should be moved to instance-rate shading (and, in fact, as every UI phase item represents a quad, the entire UI subsystem can be moved to only emit per-instance data).prepare_uinodesand similar duplicate in an ad-hoc way whatbatch_and_prepare_sorted_render_phasedoes. These systems should be removed in favor of the generic functionality.Much of the work that
prepare_uinodesand similar do should be moved to the extraction phase so that it can be retained from frame to frame.UI should support GPU preprocessing to eliminate the work that remains.
As mentioned before, this work should be a series of follow-ups.
On
many_buttons, with this patch I observed the median frame time to decrease from 48.29 ms to 45.16 ms, a 1.07× speedup. Note that the true value of this patch as far as performance is concerned isn't what the patch itself does but rather what it unlocks in the future.many_buttonsbefore and after:Overall picture after this patch for

many_buttons:Note the sizable performance increases that this patch should unlock by eliminating
extract_ui_camera_viewandqueue_uinodes, as well asprepare_uinodesin the future.