[docs] Simplify multi-instance development with direct PYTHONPATH
Replace pip install -e . --prefix=./.local approach with simpler PYTHONPATH method: - No pip install required - Code changes take effect immediately - Each worktree is completely isolated Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,20 +1,16 @@
|
|||||||
# Commands
|
# Commands
|
||||||
|
|
||||||
## Installation
|
## Running (with PYTHONPATH)
|
||||||
|
|
||||||
```bash
|
For multi-instance development, use PYTHONPATH instead of pip install:
|
||||||
pip install -e .
|
|
||||||
```
|
|
||||||
|
|
||||||
## Running
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Run example
|
# Run example
|
||||||
python example.py
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python example.py
|
||||||
|
|
||||||
# Run benchmarks
|
# Run benchmarks
|
||||||
python bench.py # Standard benchmark
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench.py
|
||||||
python bench_offload.py # CPU offload benchmark
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench_offload.py
|
||||||
```
|
```
|
||||||
|
|
||||||
## Config Defaults
|
## Config Defaults
|
||||||
|
|||||||
@@ -66,33 +66,27 @@ print("test_xxx: PASSED")
|
|||||||
|
|
||||||
## Running Tests
|
## Running Tests
|
||||||
|
|
||||||
|
Use PYTHONPATH for multi-instance isolation (no pip install needed):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Run a specific test
|
# Run a specific test
|
||||||
python tests/test_offload_engine.py
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python tests/test_offload_engine.py
|
||||||
|
|
||||||
# Run with specific GPU
|
# Run with specific GPU
|
||||||
CUDA_VISIBLE_DEVICES=0 python tests/test_ring_buffer.py
|
CUDA_VISIBLE_DEVICES=0 PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python tests/test_ring_buffer.py
|
||||||
```
|
```
|
||||||
|
|
||||||
## Benchmarks
|
## Benchmarks
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Standard GPU benchmark
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench.py
|
||||||
python bench.py
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench_offload.py
|
||||||
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python bench_vllm.py
|
||||||
# CPU offload benchmark
|
|
||||||
python bench_offload.py
|
|
||||||
|
|
||||||
# vLLM comparison benchmark
|
|
||||||
python bench_vllm.py
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Quick Verification
|
## Quick Verification
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Import test
|
# Import test
|
||||||
python -c "from nanovllm import LLM"
|
PYTHONPATH=/path/to/nano-vllm:$PYTHONPATH python -c "from nanovllm import LLM"
|
||||||
|
|
||||||
# Run offload benchmark (tests CPU-primary ring buffer mode)
|
|
||||||
python bench_offload.py
|
|
||||||
```
|
```
|
||||||
|
|||||||
48
CLAUDE.md
48
CLAUDE.md
@@ -44,45 +44,31 @@ python bench_offload.py
|
|||||||
- Running examples (`python example.py`)
|
- Running examples (`python example.py`)
|
||||||
- Any script that imports torch/cuda
|
- Any script that imports torch/cuda
|
||||||
|
|
||||||
## Local Package Installation for Multi-Instance
|
## Multi-Instance Development with PYTHONPATH
|
||||||
|
|
||||||
**CRITICAL**: After ANY code modification in the `nanovllm/` directory, you MUST reinstall the package before running tests or benchmarks:
|
**IMPORTANT**: When running multiple Claude instances on different worktrees, do NOT use `pip install -e .` globally as it will affect other instances.
|
||||||
|
|
||||||
|
**Use PYTHONPATH directly** - no pip install needed:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install -e . --prefix=./.local --no-deps
|
# Set PYTHONPATH to point to the project root directory
|
||||||
|
PYTHONPATH=/path/to/your/worktree:$PYTHONPATH python <script.py>
|
||||||
|
|
||||||
|
# Example: running tests
|
||||||
|
PYTHONPATH=/home/zijie/Code/nano-vllm:$PYTHONPATH python tests/test_needle.py
|
||||||
```
|
```
|
||||||
|
|
||||||
Then run with PYTHONPATH:
|
**Benefits**:
|
||||||
|
- No `pip install` required
|
||||||
|
- Code changes take effect immediately (no reinstall needed)
|
||||||
|
- Each worktree is completely isolated
|
||||||
|
|
||||||
|
**For shell session** (optional):
|
||||||
```bash
|
```bash
|
||||||
PYTHONPATH=./.local/lib/python3.10/site-packages:$PYTHONPATH python <script.py>
|
export PYTHONPATH=/path/to/your/worktree:$PYTHONPATH
|
||||||
|
python tests/test_needle.py # PYTHONPATH already set
|
||||||
```
|
```
|
||||||
|
|
||||||
**IMPORTANT**: When running multiple Claude instances on different worktrees, do NOT use `pip install -e .` globally as it will affect other instances. Instead, use local installation:
|
|
||||||
|
|
||||||
1. **Install to worktree-local directory**:
|
|
||||||
```bash
|
|
||||||
pip install -e . --prefix=./.local --no-deps
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Set PYTHONPATH before running any Python command**:
|
|
||||||
```bash
|
|
||||||
export PYTHONPATH=./.local/lib/python3.10/site-packages:$PYTHONPATH
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Combined example**:
|
|
||||||
```bash
|
|
||||||
# One-liner for running tests with local package
|
|
||||||
PYTHONPATH=./.local/lib/python3.10/site-packages:$PYTHONPATH python tests/test_needle.py
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note**: The Python version in the path (python3.10) should match your environment.
|
|
||||||
|
|
||||||
**CRITICAL**: After making code changes to `nanovllm/` source files, you MUST reinstall the package for changes to take effect:
|
|
||||||
```bash
|
|
||||||
pip install -e . --prefix=./.local --no-deps
|
|
||||||
```
|
|
||||||
Without reinstallation, Python will use the old cached version and your changes will NOT be reflected!
|
|
||||||
|
|
||||||
## Sparse Attention
|
## Sparse Attention
|
||||||
|
|
||||||
For sparse attention related content (block sparse attention, MInference, FlexPrefill, XAttention, AvgPool, etc.), refer to [`docs/sparse_attention_guide.md`](docs/sparse_attention_guide.md).
|
For sparse attention related content (block sparse attention, MInference, FlexPrefill, XAttention, AvgPool, etc.), refer to [`docs/sparse_attention_guide.md`](docs/sparse_attention_guide.md).
|
||||||
|
|||||||
Reference in New Issue
Block a user