feat(platform): discover out-of-tree backends via entry points#1126
feat(platform): discover out-of-tree backends via entry points#1126XHPlus wants to merge 1 commit into
Conversation
Add a `lightx2v.platform_plugins` entry-point scan to `set_ai_device()` so a chip backend can be shipped as a separate pip package and registered at runtime without editing this repo — the same out-of-tree plugin model vLLM uses. The scan runs before `init_ai_device()` (so a plugin-provided device is visible to the lookup) and before `lightx2v_platform.ops` is imported, i.e. before `registry_factory` snapshots the platform registries via `merge()`. That ordering is what lets plugin registrations reach the framework-facing registries. Safe by construction: no-op when no plugins are installed, in-tree platforms unchanged, a failing plugin is logged and skipped, and no new dependency (`importlib.metadata` is stdlib). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces support for out-of-tree platform plugins by dynamically discovering and loading them via importlib.metadata entry points during device initialization. It also updates the documentation with instructions on how to package and register these plugins. A review comment suggests adding a module-level guard to prevent loading plugins multiple times, which could otherwise cause duplicate registration errors if set_ai_device() is invoked repeatedly.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| def _load_platform_plugins(): | ||
| """Discover out-of-tree platform backends via entry points. | ||
|
|
||
| Third-party packages register under the ``lightx2v.platform_plugins`` entry | ||
| point group. Each entry point is a zero-arg callable that registers its | ||
| Device class into ``PLATFORM_DEVICE_REGISTER`` and its ops into the | ||
| ``PLATFORM_*`` op registries. | ||
|
|
||
| This runs before ``init_ai_device`` (so a plugin-provided device is visible | ||
| to the lookup) and before ``lightx2v_platform.ops`` is imported, i.e. before | ||
| ``lightx2v.utils.registry_factory`` snapshots the platform registries via | ||
| ``merge()``. That ordering is what makes plugin registrations reach the | ||
| framework-facing registries. | ||
|
|
||
| No effect when no plugins are installed; a failing plugin is logged and | ||
| skipped rather than aborting device setup. | ||
| """ | ||
| try: |
There was a problem hiding this comment.
If set_ai_device() is called multiple times (e.g., in interactive environments like Jupyter notebooks, or during unit testing), _load_platform_plugins() will be executed repeatedly. This will attempt to re-register the same devices and operators, which triggers a duplicate registration error (Exception: <key> already exists.) in Register.register(). Adding a simple module-level guard ensures plugins are only loaded once.
_plugins_loaded = False
def _load_platform_plugins():
"""Discover out-of-tree platform backends via entry points.
Third-party packages register under the ``lightx2v.platform_plugins`` entry
point group. Each entry point is a zero-arg callable that registers its
Device class into ``PLATFORM_DEVICE_REGISTER`` and its ops into the
``PLATFORM_*`` op registries.
This runs before ``init_ai_device`` (so a plugin-provided device is visible
to the lookup) and before ``lightx2v_platform.ops`` is imported, i.e. before
``lightx2v.utils.registry_factory`` snapshots the platform registries via
``merge()``. That ordering is what makes plugin registrations reach the
framework-facing registries.
No effect when no plugins are installed; a failing plugin is logged and
skipped rather than aborting device setup.
"""
global _plugins_loaded
if _plugins_loaded:
return
_plugins_loaded = True
try:
Motivation
Adding a chip backend to
lightx2v_platformcurrently requires editing in-tree files (base/__init__.pyto import theDeviceclass,ops/__init__.pyto add anelif PLATFORM == ...branch). That means every backend must be upstreamed or the user must patch the repo to use it.This PR adds an out-of-tree (OOT) plugin seam so a chip backend can be shipped as a separate pip-installable package and registered at runtime — the same mechanism vLLM uses for hardware backends via
vllm.platform_plugins.What changed
set_ai_device()now scans thelightx2v.platform_pluginsentry-point group and invokes each registered callable before initialising the device.A plugin package declares:
Then
pip install my-backend-package && PLATFORM=my_backend python lightx2v/infer.py ...— no edits to this repo.Why the scan sits inside
set_ai_device()lightx2v/utils/registry_factory.pybuilds the framework-facing registries by copying the platform staging registries at import time:merge()is a one-shot snapshot, not a live view. So a plugin's registrations are only picked up if they run before that import. The scan is placed:init_ai_device()— so a plugin-provided device (e.g. itsPLATFORMkey) is visible to the lookup;from lightx2v_platform.ops import *— i.e. beforeregistry_factorysnapshots thePLATFORM_*op tables.Safety
entry_points(group=...)returns empty; the loop is a no-op. In-tree platforms are untouched.register()raises is logged and skipped; it cannot crashset_ai_device().importlib.metadatais stdlib (3.8+); theTypeErrorbranch handles the pre-3.10 dict-like return.First consumer
lightx2v-plugin-FL(https://github.com/ModelTC/lightx2v-plugin-FL) uses this hook to expose aflagosmeta-platform: a singlePLATFORM=flagosbackend that covers many domestic chips at once by delegating compute to FlagGems and collective communication to FlagCX, instead of one hand-written kernel set per chip.Test plan
set_ai_device()for in-tree platforms (cuda,ascend_npu, ...) behaves identically.set_ai_device.pycompiles; scan is a no-op when no entry points are registered.lightx2v.platform_pluginsentry point; assert its device key reachesPLATFORM_DEVICE_REGISTERand its op keys reachlightx2v.utils.registry_factoryafterimport lightx2v.🤖 Generated with Claude Code