nano-vllm

Files

Zijie Tian 4d1e40152d ✨ feat(xattn): implement compute_chunked_prefill with ring buffer pipeline

- Copy compute_chunked_prefill implementation from FullAttentionPolicy
- Set default threshold to 0.95 for accuracy testing
- Remove debug code (sys.exit, verbose prints)
- Use ring buffer pipeline for historical block loading
- Merge with current chunk attention using flash_attn_with_lse

RULER NIAH test passed with 5/5 samples (100% accuracy).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-23 08:27:40 +08:00

policies

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

sparse

✨ feat(xattn): implement compute_chunked_prefill with ring buffer pipeline

2026-01-23 08:27:40 +08:00

__init__.py

[WIP] Before integrate the xattn operator.

2026-01-19 21:19:21 +08:00

base_manager.py

[feat] Added chunked prefill and kvcache offload mechenism.