Zijie Tian ed3c8bb4b8 🐛 fix: memory leak in XAttentionBSAPolicy select_blocks ...

Fix severe memory leak (64GB -> 4GB growth) by:
- Remove unused sparse_metadata storage (was accumulating attn_scores)
- Delete intermediate tensor list (attn_scores_list) after use
- Explicitly delete intermediate tensors before return

Before: 16GB -> 80GB during 128K prefill
After:  16GB -> 19.8GB during 128K prefill

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-23 09:30:18 +08:00

..

comm

[WIP] Added sgDMA operator for scatter kvcache communication.

2025-12-24 23:48:52 +08:00

debug

[refactor] Refactor the kvcache offload.

2026-01-04 19:37:03 +08:00

engine

Merge branch 'tzj/minference' of ssh://git.zijie-tian.site:2222/zijie-tian/nano-vllm into tzj/minference

2026-01-20 02:27:10 +08:00

kvcache

🐛 fix: memory leak in XAttentionBSAPolicy select_blocks

2026-01-23 09:30:18 +08:00

layers

♻️ refactor: move select_blocks from policy to attention layer

2026-01-23 05:21:28 +08:00

models

[claudesquad] update from 'add-llama-1' on 10 Jan 26 21:03 CST

2026-01-10 21:03:45 +08:00

ops

feat: add xattn kernels test and update testing rules

2026-01-23 03:01:25 +08:00

utils

[feat] Need to optimized with async prefetch.

2025-12-15 06:58:40 +08:00

__init__.py

better

2025-06-15 10:36:45 +08:00

config.py

[WIP] Before refactor the compute)_chunked_prefill.

2026-01-23 03:36:12 +08:00

llm.py

support tensor parallel

2025-06-15 01:31:24 +08:00

sampling_params.py

compile random sampling

2025-08-31 22:55:34 +08:00