Zijie Tian 18bc433f09 ⚡ perf: improve NVTX profiling with colored ranges and configurable slots ...

- Switch from torch.cuda.nvtx to nvtx package for colored range support
- Add color coding: blue for H2D, green for D2H decode, orange for D2H prefill
- Add --num-gpu-blocks parameter to profile_offload.sh
- Include slot count in output filename for easier comparison

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>

2026-01-27 03:42:05 +08:00

..

policies

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

sparse

♻️ refactor: unify KV cache operations through OffloadEngine

2026-01-27 02:20:59 +08:00

__init__.py

✨ feat: add configurable stride and chunk_size for XAttention BSA

2026-01-23 10:37:04 +08:00

base_manager.py

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

gpu_manager.py

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

hybrid_manager.py

🐛 fix: resolve CPU KV cache state leakage between requests

2026-01-21 01:12:21 +08:00

kernels.py

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

offload_engine.py

⚡ perf: improve NVTX profiling with colored ranges and configurable slots

2026-01-27 03:42:05 +08:00