Zijie Tian 61edb8a344 [feat] Finished offload. Still need optimize performance.

2025-12-12 02:27:40 +08:00

..

engine

[feat] Added bench_offload.py and GreedySampler.

2025-12-12 00:24:08 +08:00

kvcache

[fix] Fixed kvcache offload problem.

2025-12-12 01:35:30 +08:00

layers

[feat] Finished offload. Still need optimize performance.

2025-12-12 02:27:40 +08:00

models

[refactor] Translate into english, void Chinese due to claude.

2025-12-11 00:30:24 +08:00

utils

[fix] Fixed kvcache offload problem.

2025-12-12 01:35:30 +08:00

__init__.py

better

2025-06-15 10:36:45 +08:00

config.py

[refactor] Translate into english, void Chinese due to claude.

2025-12-11 00:30:24 +08:00

llm.py

support tensor parallel

2025-06-15 01:31:24 +08:00

sampling_params.py

compile random sampling

2025-08-31 22:55:34 +08:00