Zijie Tian
64971c8e8a
Merge branch 'zijie/fix-dist-3': Fix distributed port conflict
...
- Auto port allocation with _find_free_port() in model_runner.py
- Resource management refactor with close() + context manager in llm_engine.py
- Add tests/test_port_conflict.py and tests/run_parallel_niah.sh
- Remove docs/torch_distributed_port_issue.md (issue fixed)
- Ignore tests/data/ directory
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-12 16:27:25 +08:00
Zijie Tian
e554d5482b
[refactor] Delete unnesscessory test, and refacrtor the offload prefix cache.
2026-01-05 20:31:42 +08:00
Zijie Tian
89f8020d38
[WIP] fixing attention compute error.
2025-12-30 00:31:48 +08:00
Zijie Tian
0b6f19242d
[feat] Added chunked prefill and kvcache offload mechenism.
2025-12-10 03:47:37 +08:00
Zijie Tian
204fe2b38f
[feat] Added metric into tqdm bar.
2025-12-10 00:52:13 +08:00
GeeeekExplorer
df99418f7d
simplify
2025-08-31 20:02:51 +08:00
xiaohajiayou
054aec852d
Fix: Division-by-Zero Risk and Typo
2025-06-24 02:02:33 +08:00
Xingkai Yu
4fc764f175
Merge pull request #22 from cheunglei/use_spawn
2025-06-17 23:53:59 +08:00
cheunglei
b5ace32982
use spawn
2025-06-17 23:49:15 +08:00
GeeeekExplorer
bc0ad5a116
better
2025-06-17 23:33:38 +08:00
GeeeekExplorer
7e42fa6f63
fix
2025-06-15 13:28:29 +08:00
GeeeekExplorer
fc778a4da9
better
2025-06-15 10:36:45 +08:00
cheunglei
53b3ef2e32
support tensor parallel
2025-06-15 01:31:24 +08:00
GeeeekExplorer
4a8aa090a7
fix
2025-06-14 00:56:07 +08:00
GeeeekExplorer
f16adb729e
refactor
2025-06-12 09:41:12 +08:00
GeeeekExplorer
fee58d44e4
fix
2025-06-12 01:00:31 +08:00
GeeeekExplorer
386290d69e
refactor
2025-06-11 21:12:57 +08:00
GeeeekExplorer
b98e1ca305
fix
2025-06-10 21:25:54 +08:00
GeeeekExplorer
a5a4909e6a
init commit
2025-06-10 00:27:01 +08:00
