Skip to content

feat(platform): discover out-of-tree backends via entry points#1126

Open
XHPlus wants to merge 1 commit into
ModelTC:mainfrom
XHPlus:feat/platform-plugin-entrypoints
Open

feat(platform): discover out-of-tree backends via entry points#1126
XHPlus wants to merge 1 commit into
ModelTC:mainfrom
XHPlus:feat/platform-plugin-entrypoints

Conversation

@XHPlus

@XHPlus XHPlus commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Motivation

Adding a chip backend to lightx2v_platform currently requires editing in-tree files (base/__init__.py to import the Device class, ops/__init__.py to add an elif PLATFORM == ... branch). That means every backend must be upstreamed or the user must patch the repo to use it.

This PR adds an out-of-tree (OOT) plugin seam so a chip backend can be shipped as a separate pip-installable package and registered at runtime — the same mechanism vLLM uses for hardware backends via vllm.platform_plugins.

What changed

set_ai_device() now scans the lightx2v.platform_plugins entry-point group and invokes each registered callable before initialising the device.

A plugin package declares:

[project.entry-points."lightx2v.platform_plugins"]
my_backend = "my_backend_pkg:register"
def register():
    from . import device          # @PLATFORM_DEVICE_REGISTER("my_backend")
    from .ops import register_ops
    register_ops()

Then pip install my-backend-package && PLATFORM=my_backend python lightx2v/infer.py ... — no edits to this repo.

Why the scan sits inside set_ai_device()

lightx2v/utils/registry_factory.py builds the framework-facing registries by copying the platform staging registries at import time:

ATTN_WEIGHT_REGISTER.merge(PLATFORM_ATTN_WEIGHT_REGISTER)
MM_WEIGHT_REGISTER.merge(PLATFORM_MM_WEIGHT_REGISTER)
...

merge() is a one-shot snapshot, not a live view. So a plugin's registrations are only picked up if they run before that import. The scan is placed:

  • before init_ai_device() — so a plugin-provided device (e.g. its PLATFORM key) is visible to the lookup;
  • before from lightx2v_platform.ops import * — i.e. before registry_factory snapshots the PLATFORM_* op tables.

Safety

  • No behaviour change without pluginsentry_points(group=...) returns empty; the loop is a no-op. In-tree platforms are untouched.
  • Failure isolation — a plugin whose register() raises is logged and skipped; it cannot crash set_ai_device().
  • No new dependencyimportlib.metadata is stdlib (3.8+); the TypeError branch handles the pre-3.10 dict-like return.

First consumer

lightx2v-plugin-FL (https://github.com/ModelTC/lightx2v-plugin-FL) uses this hook to expose a flagos meta-platform: a single PLATFORM=flagos backend that covers many domestic chips at once by delegating compute to FlagGems and collective communication to FlagCX, instead of one hand-written kernel set per chip.

Test plan

  • No-plugin regression: set_ai_device() for in-tree platforms (cuda, ascend_npu, ...) behaves identically.
  • set_ai_device.py compiles; scan is a no-op when no entry points are registered.
  • Plugin discovery (reviewer-runnable): install a stub package exposing a lightx2v.platform_plugins entry point; assert its device key reaches PLATFORM_DEVICE_REGISTER and its op keys reach lightx2v.utils.registry_factory after import lightx2v.

🤖 Generated with Claude Code

Add a `lightx2v.platform_plugins` entry-point scan to `set_ai_device()` so a
chip backend can be shipped as a separate pip package and registered at runtime
without editing this repo — the same out-of-tree plugin model vLLM uses.

The scan runs before `init_ai_device()` (so a plugin-provided device is visible
to the lookup) and before `lightx2v_platform.ops` is imported, i.e. before
`registry_factory` snapshots the platform registries via `merge()`. That
ordering is what lets plugin registrations reach the framework-facing
registries.

Safe by construction: no-op when no plugins are installed, in-tree platforms
unchanged, a failing plugin is logged and skipped, and no new dependency
(`importlib.metadata` is stdlib).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for out-of-tree platform plugins by dynamically discovering and loading them via importlib.metadata entry points during device initialization. It also updates the documentation with instructions on how to package and register these plugins. A review comment suggests adding a module-level guard to prevent loading plugins multiple times, which could otherwise cause duplicate registration errors if set_ai_device() is invoked repeatedly.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +8 to +25
def _load_platform_plugins():
"""Discover out-of-tree platform backends via entry points.

Third-party packages register under the ``lightx2v.platform_plugins`` entry
point group. Each entry point is a zero-arg callable that registers its
Device class into ``PLATFORM_DEVICE_REGISTER`` and its ops into the
``PLATFORM_*`` op registries.

This runs before ``init_ai_device`` (so a plugin-provided device is visible
to the lookup) and before ``lightx2v_platform.ops`` is imported, i.e. before
``lightx2v.utils.registry_factory`` snapshots the platform registries via
``merge()``. That ordering is what makes plugin registrations reach the
framework-facing registries.

No effect when no plugins are installed; a failing plugin is logged and
skipped rather than aborting device setup.
"""
try:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If set_ai_device() is called multiple times (e.g., in interactive environments like Jupyter notebooks, or during unit testing), _load_platform_plugins() will be executed repeatedly. This will attempt to re-register the same devices and operators, which triggers a duplicate registration error (Exception: <key> already exists.) in Register.register(). Adding a simple module-level guard ensures plugins are only loaded once.

_plugins_loaded = False


def _load_platform_plugins():
    """Discover out-of-tree platform backends via entry points.

    Third-party packages register under the ``lightx2v.platform_plugins`` entry
    point group. Each entry point is a zero-arg callable that registers its
    Device class into ``PLATFORM_DEVICE_REGISTER`` and its ops into the
    ``PLATFORM_*`` op registries.

    This runs before ``init_ai_device`` (so a plugin-provided device is visible
    to the lookup) and before ``lightx2v_platform.ops`` is imported, i.e. before
    ``lightx2v.utils.registry_factory`` snapshots the platform registries via
    ``merge()``. That ordering is what makes plugin registrations reach the
    framework-facing registries.

    No effect when no plugins are installed; a failing plugin is logged and
    skipped rather than aborting device setup.
    """
    global _plugins_loaded
    if _plugins_loaded:
        return
    _plugins_loaded = True

    try:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant