zijie-tian/nano-vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[fix] Fixed needle test bug.

Browse Source
This commit is contained in:
Zijie Tian
2026-01-05 18:34:09 +08:00
parent d623043a3c
commit 054aaff403
5 changed files with 8 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
CLAUDE.md
View File

@@ -302,7 +302,7 @@ def _merge_output_kernel(...):
| Parameter | Default | Notes |
|-----------|---------|-------|
| `kvcache_block_size` | 4096 | Tokens per block |
| `kvcache_block_size` | 1024 | Tokens per block |
| `max_num_batched_tokens` | 16384 | Set = max_model_len for long context |
| `gpu_memory_utilization` | 0.9 | GPU memory fraction |
| `enable_cpu_offload` | False | Enable for long context |