zijie-tian/nano-vllm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #24 from jinghuan-Chen/fix/Release-CUDA-Graphs-resource-before-exit

Browse Source
This commit is contained in:
Xingkai Yu
2025-06-18 17:15:28 +08:00
committed by GitHub
parent 4fc764f175 ffafaeb133
commit fa0078174e
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
nanovllm/engine/model_runner.py
View File

@@ -53,6 +53,9 @@ class ModelRunner:
dist.barrier() dist.barrier()
if self.rank == 0: if self.rank == 0:
self.shm.unlink() self.shm.unlink()
if not self.enforce_eager:
del self.graphs, self.graph_pool
torch.cuda.synchronize()
dist.destroy_process_group() dist.destroy_process_group()
def loop(self): def loop(self):