Updated example file for ai200 runs

[asmigosw]

Overview

Adds a production-ready example demonstrating MiniMax-M3 VLM decode-only inference on AI200 servers with replicate KV-head optimization.

What's Updated

📄 Files

• examples/text_generation/minimax_m3_decode_only.py - Main inference script
• examples/text_generation/README_MINIMAX_M3_AI200.md - Comprehensive documentation

✨ Key Features

• Replicate KV-head optimization (num_replicate_kv_heads: 8) for AI200 hardware
• Decode-only mode (PL=1) for efficient token-by-token generation
• Mixed-precision computation with MXFP6 matmul and MXINT8 KV cache
• Multi-device parallelism supporting up to 24 AI200 devices
• Flexible CLI with comprehensive argument support

Usage

Quick Start

# Install dependencies
pip install transformers --upgrade

# Run with defaults
python examples/text_generation/minimax_m3_decode_only.py

# Custom configuration
python examples/text_generation/minimax_m3_decode_only.py \
    --ctx-len 2048 \
    --generation-len 64 \
    --prompt "Your prompt here"

Replicate KV-Head Configuration

# Applied in model initialization and compilation
qaic_config={"num_replicate_kv_heads": 8}