Zijie Tian 82ed34fc2d [opt] optimize nanovllm performance compareable with vllm.

2025-12-25 03:47:07 +08:00

..

policies

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

sparse

[feat] Added sparse KVcache feature, NEED VERIFY.

2025-12-22 08:51:02 +08:00

__init__.py

[WIP] remove num_prefetch_blocks varible.

2025-12-24 18:22:26 +08:00

base_manager.py

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

chunked_attention.py

[WIP] replace merge attention with triton kernel.

2025-12-25 01:07:05 +08:00

gpu_manager.py

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

hybrid_manager.py

[WIP] remove num_prefetch_blocks varible.

2025-12-24 18:22:26 +08:00

kernels.py

[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00

offload_engine.py

[opt] optimize nanovllm performance compareable with vllm.

2025-12-25 03:47:07 +08:00