Zijie Tian 8fd25d72d7 Merge perf_opt-1 and perf_opt-2 branches ...

Combines two performance optimization features:
- perf_opt-1: Cross-layer pipeline for decode (double-buffered layer cache)
- perf_opt-2: Per-layer prefill buffer for async offload

Both features are complementary and improve CPU offload performance.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-07 06:03:44 +08:00

..

comm

[WIP] Added sgDMA operator for scatter kvcache communication.

2025-12-24 23:48:52 +08:00

debug

[refactor] Refactor the kvcache offload.

2026-01-04 19:37:03 +08:00

engine

Merge perf_opt-1 and perf_opt-2 branches

2026-01-07 06:03:44 +08:00

kvcache

Merge perf_opt-1 and perf_opt-2 branches

2026-01-07 06:03:44 +08:00

layers

Merge perf_opt-1 and perf_opt-2 branches

2026-01-07 06:03:44 +08:00

models

[refactor] Translate into english, void Chinese due to claude.

2025-12-11 00:30:24 +08:00

utils

[feat] Need to optimized with async prefetch.

2025-12-15 06:58:40 +08:00

__init__.py

better

2025-06-15 10:36:45 +08:00

config.py

[feat] Added Quest Sparsity Policy.

2026-01-07 03:29:21 +08:00

llm.py

support tensor parallel

2025-06-15 01:31:24 +08:00

sampling_params.py

compile random sampling

2025-08-31 22:55:34 +08:00