[feat] Added chunked prefill and kvcache offload mechenism.

2025-12-10 03:47:37 +08:00
parent 204fe2b38f
commit 0b6f19242d
25 changed files with 4414 additions and 61 deletions
--- a/tests/init.py
+++ b/tests/init.py
@@ -0,0 +1 @@
+"""Test suite for nano-vllm KV cache offload."""
				`@@ -0,0 +1 @@`
				`"""Test suite for nano-vllm KV cache offload."""`