Skip to content

feat: add configurable Qwen cache types and early cache scheduling - #2045

Merged
leejet merged 2 commits into
masterfrom
feat/qwen-prefix-cache-memory
Sep 24, 2026
Merged

leejet merged 2 commits into
masterfrom
feat/qwen-prefix-cache-memory

Conversation

@leejet

@leejet leejet commented Sep 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add qwen_image_2_1_prefix_cache_type with automatic, floating-point, and supported quantized storage types using the existing type parser.
  • Automatically use FP16 only when every attention operation selects Flash Attention; rebuild the prefix in FP32 if attention falls back.
  • Centralize early cache-output scheduling in GGMLRunner and migrate Qwen, SenseNova, ControlNet, and Wan/LTX VAE callers to avoid retaining large source activations until graph end.

Related Issue / Discussion

Fix #2040.

Additional Information

N/A

Checklist

@leejet
leejet merged commit 740c7ae into master Sep 24, 2026
9 checks passed
@leejet
leejet deleted the feat/qwen-prefix-cache-memory branch September 24, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen Image 2.1 prefix cache: store K/V as F16 when flash attention is on (half the VRAM, same output)

1 participant