nano-vllm

Files

Zijie Tian ed3c8bb4b8 🐛 fix: memory leak in XAttentionBSAPolicy select_blocks

Fix severe memory leak (64GB -> 4GB growth) by:
- Remove unused sparse_metadata storage (was accumulating attn_scores)
- Delete intermediate tensor list (attn_scores_list) after use
- Explicitly delete intermediate tensors before return

Before: 16GB -> 80GB during 128K prefill
After:  16GB -> 19.8GB during 128K prefill

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-23 09:30:18 +08:00

policies

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

sparse

🐛 fix: memory leak in XAttentionBSAPolicy select_blocks

2026-01-23 09:30:18 +08:00

__init__.py

[WIP] Before integrate the xattn operator.

2026-01-19 21:19:21 +08:00

base_manager.py

[feat] Added chunked prefill and kvcache offload mechenism.