zijie-tian/nano-vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[bench] Added vllm vs nano-vllm bench.

Browse Source
This commit is contained in:
Zijie Tian
2025-12-10 00:44:57 +08:00
parent 2f21442653
commit 761929390e
4 changed files with 183 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
nanovllm/layers/attention.py
View File

@@ -3,7 +3,7 @@ from torch import nn
import triton
import triton.language as tl
from flash_attn import flash_attn_varlen_func, flash_attn_with_kvcache
from flash_attn.flash_attn_interface import flash_attn_varlen_func, flash_attn_with_kvcache
from nanovllm.utils.context import get_context