Zijie Tian
|
484d0de9f9
|
[feat] Added debug hook to offload_engine.py.
|
2025-12-31 19:44:39 +08:00 |
|
Zijie Tian
|
89f8020d38
|
[WIP] fixing attention compute error.
|
2025-12-30 00:31:48 +08:00 |
|
Zijie Tian
|
782437c486
|
[WIP] remove num_prefetch_blocks varible.
|
2025-12-24 18:22:26 +08:00 |
|
Zijie Tian
|
1907b625b6
|
[refactor] Remove legacy mode path.
|
2025-12-22 20:17:56 +08:00 |
|
Zijie Tian
|
051f2295c9
|
[feat] Added sparse KVcache feature, NEED VERIFY.
|
2025-12-22 08:51:02 +08:00 |
|
Zijie Tian
|
b8b6478506
|
[feat] Need to optimized with async prefetch.
|
2025-12-15 06:58:40 +08:00 |
|
Zijie Tian
|
1081ab51ea
|
[refactor] Refactor offload code to multi-chunk.
|
2025-12-15 01:13:58 +08:00 |
|
Zijie Tian
|
60d24f7c12
|
[feat] Added bench_offload.py and GreedySampler.
|
2025-12-12 00:24:08 +08:00 |
|
Zijie Tian
|
babfa17354
|
[refactor] Translate into english, void Chinese due to claude.
|
2025-12-11 00:30:24 +08:00 |
|
Zijie Tian
|
e85c2b4776
|
[fix] Fixed kvcache offload bugs.
|
2025-12-10 22:34:00 +08:00 |
|
Zijie Tian
|
190df5f70d
|
[refactor] Refactor current gpu and cpu block allocation strategy.
|
2025-12-10 21:23:31 +08:00 |
|
Zijie Tian
|
87055cc5ce
|
[refactor] Implement real chunked prefill mechenism.
|
2025-12-10 18:34:01 +08:00 |
|
Zijie Tian
|
0b6f19242d
|
[feat] Added chunked prefill and kvcache offload mechenism.
|
2025-12-10 03:47:37 +08:00 |
|