Zijie Tian e09a2a5b10 ✨ feat: add Qwen2/2.5 model support ...

Separate Qwen2 from Qwen3 implementation:
- Qwen2: Uses QKV bias, no QK norm
- Qwen3: Has optional QK norm when no bias

Tested with Qwen2.5-7B-Instruct-1M, RULER niah_single_1 passed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-28 13:44:32 +08:00

7.5 KiB

Executable File

Raw Blame History

View Raw