# Benchmark Suite - Image Validation V3

Run performance benchmarks for Image Validation V3 components.

## Prerequisites

```bash
pip install Pillow numpy
pip install torch torchvision  # For CLIP benchmarks
pip install git+https://github.com/openai/CLIP.git  # For CLIP benchmarks
pip install lmdb  # For cache benchmarks
```

## Available Benchmarks

### 1. pHash Backend (`benchmark_phash.py`)
Tests DCT-based perceptual hashing performance.

```bash
cd c:\Users\terru\OneDrive\Documents\BranchPy\BranchPyApp
python -m branchpy.analysis.image_validation.benchmarks.benchmark_phash
```

**Expected Results**:
- Standard Mode (8×8): 50-100 images/sec
- Fast Mode (4×4): 250-500 images/sec
- Speedup: ~5x

### 2. CLIP Backend (`benchmark_clip.py`)
Tests OpenAI CLIP (ViT-B/32) embedding extraction.

```bash
python -m branchpy.analysis.image_validation.benchmarks.benchmark_clip
```

**Expected Results**:
- GPU (CUDA): 5-10 images/sec
- CPU: 0.5-1 images/sec
- Batch mode: 10-20 images/sec (GPU only)

### 3. LMDB Cache (`benchmark_cache.py`)
Tests cache read/write latency and throughput.

```bash
python -m branchpy.analysis.image_validation.benchmarks.benchmark_cache
```

**Target Performance**:
- Read latency: <1ms per entry
- Write latency: <5ms per entry
- Read throughput: >1000 entries/sec
- Write throughput: >200 entries/sec

### 4. End-to-End Pipeline (`benchmark_e2e.py`)
Tests complete validation workflow.

```bash
python -m branchpy.analysis.image_validation.benchmarks.benchmark_e2e
```

**Expected Results**:
- pHash (cold cache): 30-50 images/sec
- pHash (warm cache): 80-150 images/sec
- pHash Fast Mode: 200-400 images/sec

## Running All Benchmarks

```bash
cd c:\Users\terru\OneDrive\Documents\BranchPy\BranchPyApp

# Run pHash benchmark
python -m branchpy.analysis.image_validation.benchmarks.benchmark_phash

# Run cache benchmark
python -m branchpy.analysis.image_validation.benchmarks.benchmark_cache

# Run end-to-end benchmark
python -m branchpy.analysis.image_validation.benchmarks.benchmark_e2e

# Run CLIP benchmark (if dependencies installed)
python -m branchpy.analysis.image_validation.benchmarks.benchmark_clip
```

## Interpreting Results

### Throughput
- **Good**: >50 images/sec (pHash), >5 images/sec (CLIP GPU)
- **Acceptable**: 20-50 images/sec (pHash), 1-5 images/sec (CLIP)
- **Poor**: <20 images/sec (pHash), <1 images/sec (CLIP)

### Cache Latency
- **Excellent**: <0.5ms read, <2ms write
- **Good**: <1ms read, <5ms write (targets)
- **Acceptable**: <5ms read, <10ms write
- **Poor**: >5ms read, >10ms write

### Cache Hit Rate (E2E Benchmark)
- **Excellent**: >95% (second run)
- **Good**: >90%
- **Acceptable**: >80%
- **Poor**: <80%

## Troubleshooting

### Slow Performance
1. **Check GPU availability**: CLIP should use CUDA/Metal if available
2. **Verify cache enabled**: `use_cache=True` in validator
3. **Check disk speed**: LMDB performance depends on SSD vs HDD
4. **Reduce image size**: Benchmarks use 512×512, production may vary

### Import Errors
- **NumPy**: `pip install numpy`
- **Pillow**: `pip install Pillow`
- **LMDB**: `pip install lmdb`
- **PyTorch**: `pip install torch torchvision`
- **CLIP**: `pip install git+https://github.com/openai/CLIP.git`

### Out of Memory (CLIP)
- Reduce batch size in `benchmark_clip.py` (line 55)
- Use CPU instead of GPU: `device="cpu"` in CLIPBackend
- Use smaller images: `create_test_images(..., size=(256, 256))`

## Benchmark Comparison Table

| Backend | Mode | Device | Images | Time | Throughput |
|---------|------|--------|--------|------|------------|
| pHash | Standard (8×8) | CPU | 100 | ~2s | 50 img/s |
| pHash | Fast (4×4) | CPU | 100 | ~400ms | 250 img/s |
| CLIP | ViT-B/32 | GPU (RTX 3060) | 50 | ~8s | 6 img/s |
| CLIP | ViT-B/32 | CPU (i7-10700) | 50 | ~50s | 1 img/s |
| Cache | Read | SSD | 100 | ~50ms | 2000/s |
| Cache | Write | SSD | 100 | ~200ms | 500/s |

## Notes

- All benchmarks create temporary test images automatically
- Results may vary based on hardware (CPU, GPU, disk speed)
- CLIP benchmarks require CUDA/Metal for optimal performance
- Cache benchmarks require LMDB installation

---

**Last Updated**: 2025-01-XX  
**Version**: v0.9.0  
**Status**: Complete
