AI Radar

中文文档

AI Agent Skills Discovery & Comparison Platform.

Discover, compare, and evaluate AI Agent Skills from GitHub. Supports both individual skills and plugins (multi-skill repos) as first-class comparison units.

Features

• 🔍 Discovery — Automated GitHub SKILL.md crawling and indexing (4000+ skills)
• 📦 Plugin System — Multi-skill repos aggregated as "super skills" for framework-level comparison
• 📊 Comparison — Side-by-side comparison with radar charts (skills or plugins)
• 🔎 Search — Unified semantic search: skills + plugins mixed in one result set
• ⭐ Scoring — 6-dimension rule scoring at index time + LLM evaluation at search time
• 📤 Submission — User-submitted skills with 3-tier validation pipeline

Database Snapshot (Optional)

To skip the indexing step and load pre-built data instead:

1. Download the latest airoadar-db-*.tar.gz from Releases.
2. Restore it:

   tar -xzf airoadar-db-*.tar.gz
   pg_restore --no-owner --no-acl -d your_database airoadar-db.dump

3. Start the server as usual.

The snapshot contains all indexed skills, plugins, embeddings, and rule scores. LLM-derived lazy scores are not included — they regenerate on first search.

Data Refresh & Indexing

The README's "Quick Start" path assumes you restore a database snapshot. To populate the database yourself — initial crawl, incremental refresh, or backfill after schema changes — use the scripts in scripts/:

# Discover + index NEW repos (default min_stars=500).
# Skips repos already in the `repos` or `skills` tables.
PYTHONPATH=src python scripts/discover_and_index.py

# Incremental refresh: also re-scan repos whose last_indexed_at
# is older than --update-days (default 30d, top --update-limit by stars).
PYTHONPATH=src python scripts/discover_and_index.py --update

# Preview without writing.
PYTHONPATH=src python scripts/discover_and_index.py --dry-run

# Index a specific set of repos (edit the REPOS list at the top of the file).
PYTHONPATH=src python scripts/index_repos.py

How the pipeline works (scripts/discover_and_index.py):

1. Discover — search GitHub by star bucket × 10 keyword queries (SKILL.md in:readme, claude skill, mcp skill, …), then bucket-validate stars client-side (GitHub sort occasionally overflows).
2. Index — for each repo, fetch the git tree, parse every SKILL.md, generate a capability embedding, write a Skill + FreshnessSnapshot, then run scorer_v2_rules.compute_and_save and mark rejected if it fails passes_filter.
3. Dedup — repos already in the repos table (or with any skill in skills.repo_url) are skipped before any API call.

Built-in pacing: time.sleep(0.3) between skills, 1s between repos, 2s between search queries — safe to run unattended.

Other useful scripts in scripts/:

Script	Purpose
refresh_v2_rule_scores.py	Recompute 6-dim rule scores without re-fetching
recompute_plugin_embeddings.py	Rebuild plugin mean-pool embeddings
rescan_*.py	Backfill specific star brackets (1k–3k, 3k–10k, …)
backfill_repo_id.py / backfill_is_original.py	One-shot column backfills

Quick Start (Docker)

Requires Docker Desktop.

git clone https://github.com/bitdata/ai-radar.git
cd ai-radar

# Configure environment
cp .env.example .env
# Edit .env with your GITHUB_TOKEN and API keys

# Start services
docker compose up -d

# The app is available at http://localhost:8000

Quick Start (Source)

Requires Python 3.11+ and PostgreSQL with pgvector.

1. Install PostgreSQL with pgvector

# Option A: Docker (recommended)
docker compose up -d db

# Option B: Manual install
# Install PostgreSQL 15+, then:
# CREATE EXTENSION IF NOT EXISTS vector;

2. Clone and install dependencies

git clone https://github.com/bitdata/ai-radar.git
cd ai-radar

# Option A: Using uv (recommended)
uv sync

# Option B: Using pip
python -m venv .venv
source .venv/bin/activate  # Linux/macOS
# .venv\Scripts\activate   # Windows
pip install -e ".[dev]"

# Option C: Using conda
conda create -n ai-radar python=3.12
conda activate ai-radar
pip install -e ".[dev]"

3. Configure environment

cp .env.example .env

Edit .env with your settings:

Variable	Required	Description
DATABASE_URL	Yes	PostgreSQL connection string
GITHUB_TOKEN	Yes	GitHub personal access token
ANTHROPIC_API_KEY	For scoring	LLM API key for capability extraction and scoring
LLM_BASE_URL	No	Custom LLM API base URL
LLM_MODEL	No	LLM model name (default: MiniMax-M3)
OPENAI_API_KEY	For embedding	API key for embedding generation (Ali DDS DashScope)
OPENAI_BASE_URL	No	Embedding API base URL
EMBEDDING_MODEL	No	Embedding model name
SCORING_MODE	No	old (legacy), new (lazy), dual (both, for validation)

4. Run database migrations

# Using uv
uv run alembic upgrade head

# Using pip/conda
python -m alembic upgrade head

5. Start the server

# Using uv
uv run uvicorn ai_radar.main:app --reload --port 8000

# Using pip/conda
uvicorn ai_radar.main:app --reload --port 8000

Open http://localhost:8000 in your browser.

Project Structure

ai-radar/
├── src/ai_radar/
│   ├── api/                  # FastAPI routes
│   │   ├── skills.py         # Skills CRUD + search + scoring
│   │   ├── compare.py        # Side-by-side comparison (skills + plugins)
│   │   └── articles.py       # Published articles
│   ├── db/
│   │   ├── models.py         # SQLAlchemy models (Skill, Plugin, etc.)
│   │   └── migrations/       # Manual migrations (001-007)
│   ├── templates/            # Jinja2 + HTMX templates
│   │   ├── plugins/          # Plugin detail page
│   │   └── skills/           # Skill detail page
│   ├── lazy_scorer.py        # Search-time LLM evaluation (spec v3)
│   ├── plugin_aggregator.py  # Plugin → "super skill" aggregation
│   ├── scorer_v2_rules.py    # 6-dimension rule scoring (index time)
│   ├── capability.py         # Embedding + unified search (skills + plugins)
│   ├── skill_parser.py       # SKILL.md parser (YAML/JSON/markdown)
│   ├── submission_handler.py # User submission validation
│   ├── github_client.py      # GitHub API client with caching
│   └── config.py             # Settings from environment
├── tests/                    # Test suite (204 unit tests)
├── docs/                     # Design specs
├── scripts/                  # Utility scripts
│   ├── discover_and_index.py # Auto-discover + batch index repos
│   ├── index_repos.py        # Index specific repos
│   └── migrate_plugins.py    # Plugin data migration
├── docker-compose.yml        # Docker services
└── pyproject.toml            # Project metadata

Running Tests

# Using uv
uv run pytest tests/ -v

# Using pip/conda
pytest tests/ -v

# With coverage
pytest tests/ --cov=src/ai_radar --cov-report=term-missing

Internationalization

The UI supports English (default) and Simplified Chinese. Switch via the top-right language toggle; the choice is stored in a lang cookie (1 year TTL). The ?lang=zh URL parameter overrides the cookie for the current request. Accept-Language header is used as a fallback for first-time visitors.

Translation files live in src/ai_radar/translations/ (Python dicts). Adding a new UI string requires a corresponding key in both en.py and zh.py — a test (tests/unit/test_translation_coverage.py) enforces this at startup.

Scoring System

Rule Scoring (index time, 0 LLM)

6 dimensions computed for every skill at index time:

• Maintenance — stars, commit recency, issue ratio, contributors
• License — permissiveness lookup table
• Security — pattern matching for dangerous commands
• Compatibility — multi-agent support detection
• Documentation — length, examples, limitations, install guide
• Solidity — repo health gate (rejects if < 1.0)

Lazy Scoring (search time, 1 LLM call)

On-demand evaluation at search time:

1. pgvector recall → top 200 candidates
2. Name dedup + hybrid sort → top 50
3. LLM call: pick ≤10 relevant + propose 5-9 query-specific dimensions + rank
4. Post-process: drop low-confidence dimensions

Plugin System

Multi-skill repos (e.g., obra/superpowers with 15 skills) are aggregated into "super skills":

• Description: concatenated from all child skills
• Rule scores: averaged across all child skills
• Embedding: mean-pooled from child skill embeddings
• Search: plugins appear alongside individual skills in unified results (📦 marker)

Tech Stack

• Backend: FastAPI + SQLAlchemy + Alembic
• Database: PostgreSQL + pgvector
• Frontend: Jinja2 templates + HTMX + Tailwind CSS
• LLM: Anthropic-compatible API (MiniMax)
• Embedding: OpenAI-compatible API (Ali DDS DashScope)

License

Apache License 2.0. This software may also be made available under different terms or with additional capabilities; contact the maintainers for details.

See LICENSE for the full license text.