zijie-tian/nano-vllm

Files

Zijie Tian bbbfd1e7da [docs] Simplify multi-instance development with direct PYTHONPATH

Replace pip install -e . --prefix=./.local approach with simpler PYTHONPATH method:
- No pip install required
- Code changes take effect immediately
- Each worktree is completely isolated

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-01-08 04:51:55 +08:00

527 B

Raw Blame History

Commands

Running (with PYTHONPATH)

For multi-instance development, use PYTHONPATH instead of pip install:

# Run example
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python example.py

# Run benchmarks
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench.py
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench_offload.py

Config Defaults

max_num_batched_tokens: 16384
max_num_seqs: 512
kvcache_block_size: 4096
gpu_memory_utilization: 0.9
enforce_eager: False (enables CUDA graphs)