# Model Card: Bitrater Deep Learning Classifier

## Model Details

- **Architecture**: Two-stage CNN + BiLSTM with multi-head attention
- **Task**: 7-class audio bitrate classification
- **Version**: 1.0
- **Files**: `bitrater/models/stage1_cnn.onnx` (1.7 MB), `bitrater/models/stage2_seq.onnx` (3.1 MB)
- **Inference**: ONNX Runtime (no PyTorch required)
- **Training**: PyTorch (optional `[training]` extra)
- **License**: MIT (same as project)

## Architecture

### Stage 1: CNN Feature Extractor (~430K parameters)

Extracts a 128-dimensional feature vector from each 2-second spectrogram window.

| Layer | Output Shape | Notes |
|-------|-------------|-------|
| Conv2d(1, 32) + BN + ReLU + MaxPool2d(2) | (32, 64, T/2) | |
| Conv2d(32, 64) + BN + ReLU + MaxPool2d(2) | (64, 32, T/4) | |
| Conv2d(64, 128) + BN + ReLU + MaxPool2d(2) | (128, 16, T/8) | |
| Conv2d(128, 256) + BN + ReLU + MaxPool2d(2) | (256, 8, T/16) | |
| AdaptiveAvgPool2d(1) + Linear(256, 128) | (128,) | Feature vector |

Dropout (p=0.2) after each conv block.

### Stage 2: Sequence Classifier (~700K parameters)

Processes sequences of 48 CNN feature vectors with auxiliary features.

- **BiLSTM**: 2-layer, hidden_dim=128, bidirectional (output: 256-dim per timestep)
- **Multi-head attention pooling**: 4 heads over LSTM outputs
- **Auxiliary input**: 211-dim vector (181 SVM spectral features + 30 global modulation DCT features)
- **Classifier head**: Linear(256+211, 256) -> ReLU -> Linear(256, 7)
- **VBR auxiliary head**: Linear(256, 32) -> ReLU -> Linear(32, 1)

### Input Representation

**Dual-band spectrogram** (128 frequency bins x T time frames):
- Bins 0-63: Mel-scale filterbank, 0-16 kHz (captures overall spectral content)
- Bins 64-127: Linear-scale, 16-22 kHz with ~94 Hz resolution (captures MP3 compression cutoffs)

This gives 3x better frequency resolution in the critical 16-22 kHz region compared to a standard 128-bin mel spectrogram, enabling precise detection of:
- CBR 192 cutoff at ~18.6 kHz
- CBR 320 cutoff at ~20.5 kHz
- VBR's variable cutoff behavior vs CBR's fixed cutoff

### Inference Pipeline

For each file:
1. Load audio at 44.1 kHz mono
2. Compute shared STFT (n_fft=2048, hop=512)
3. Extract dual-band spectrograms for 2-second windows (1-second hop)
4. CNN forward pass on all needed windows -> 128-dim feature vectors
5. Extract global modulation features (2D DCT over log-mel, 30 coefficients)
6. Extract SVM spectral features (181-dim PSD/cutoff/temporal/artifact vector)
7. Sliding window over sequences of 48 CNN features + auxiliary features
8. Average softmax probabilities across all sequences -> final prediction

Typical inference time: 2-5 seconds per file on CPU.

## Training

### Dataset

Training data was generated by transcoding a collection of ~2,400 lossless FLAC files (85 GB) to MP3 at each target bitrate using LAME:

| Class | Encoding | Files | Description |
|-------|----------|-------|-------------|
| 128 | `lame -b 128` | ~2,400 | CBR 128 kbps |
| V2 | `lame -V 2` | ~2,400 | VBR preset 2 (~170-210 kbps) |
| 192 | `lame -b 192` | ~2,400 | CBR 192 kbps |
| V0 | `lame -V 0` | ~2,400 | VBR preset 0 (~220-260 kbps) |
| 256 | `lame -b 256` | ~2,400 | CBR 256 kbps |
| 320 | `lame -b 320` | ~2,400 | CBR 320 kbps |
| LOSSLESS | (original) | ~2,400 | Source FLAC files |

Total: ~16,800 files across 7 classes. Split: 80% train, 10% validation, 10% test.

Source material spans diverse genres and recording conditions.

### Training Procedure

**Stage 1 (CNN warmup):**
- Train `WindowClassifier` on individual 2-second spectrogram windows
- 25 epochs, OneCycleLR (max_lr=1e-3), batch size 64
- Focal loss (gamma=2.0) with class weighting [1.0, 2.0, 2.0, 1.5, 1.0, 1.0, 1.0]
- Label smoothing (0.05)
- Max 10 random windows sampled per file per epoch

**Stage 2 (sequence training):**
- Freeze CNN, train `SequenceClassifier` on sequences of 48 CNN feature vectors
- 60 epochs, OneCycleLR (max_lr=3e-4), batch size 32
- Same focal loss and class weighting as Stage 1
- 4 random sequences sampled per file per epoch
- Early stopping with patience 15 on validation accuracy

**Hardware**: Trained on NVIDIA GTX 1660 Super (6 GB VRAM), FP32 only (no Tensor Cores).

### Key Design Decisions

- **Dual-band filterbank**: Standard mel spectrograms lose resolution above 16 kHz. The linear high-frequency band provides precise cutoff detection.
- **Two-stage training**: Separates representation learning (CNN on windows) from temporal reasoning (BiLSTM on sequences).
- **Focal loss with class weighting**: V2 and 192 kbps are inherently hard to separate (overlapping bitrate ranges), so they receive 2x class weight.
- **SVM features as auxiliary input**: The hand-crafted SVM features (PSD bands, cutoff detection, temporal statistics) provide complementary information to the learned CNN features.
- **Global modulation features**: 2D DCT over the full-file spectrogram captures long-range spectral patterns that distinguish VBR's gradually varying lowpass filter from CBR's fixed cutoff.
- **Variance features disabled**: Ablation study showed temporal variance/range features did not improve accuracy when SVM+global modulation features are present.

## Performance

**Overall accuracy: 98.42%** (file-level evaluation on held-out test set, 1,391 files)

### Per-Class Metrics

| Class | Precision | Recall | F1 Score | Support |
|-------|-----------|--------|----------|---------|
| 128 | 98.5% | 100.0% | 99.2% | 195 |
| V2 | 98.4% | 97.3% | 97.8% | 184 |
| 192 | 97.7% | 98.2% | 97.9% | 219 |
| V0 | 98.1% | 97.7% | 97.9% | 217 |
| 256 | 99.4% | 99.4% | 99.4% | 170 |
| 320 | 97.7% | 98.6% | 98.1% | 213 |
| LOSSLESS | 99.5% | 97.9% | 98.7% | 193 |

### Confusion Matrix

Rows are true labels, columns are predictions.

| | 128 | V2 | 192 | V0 | 256 | 320 | LOSSLESS |
|----------|----:|---:|----:|---:|----:|----:|---------:|
| **128** | 195 | 0 | 0 | 0 | 0 | 0 | 0 |
| **V2** | 1 | 179 | 3 | 1 | 0 | 0 | 0 |
| **192** | 1 | 3 | 215 | 0 | 0 | 0 | 0 |
| **V0** | 0 | 0 | 0 | 212 | 0 | 4 | 1 |
| **256** | 0 | 0 | 1 | 0 | 169 | 0 | 0 |
| **320** | 1 | 0 | 0 | 1 | 1 | 210 | 0 |
| **LOSSLESS** | 0 | 0 | 1 | 2 | 0 | 1 | 189 |

Total errors: 22/1,391 (1.58%). V2/192 cross-confusion: 6 files.

### Known Limitations

- **V2/192 confusion**: These two classes have the most overlap — V2 VBR averages 170-210 kbps with a variable lowpass filter, while CBR 192 has a fixed ~18.6 kHz cutoff. The model achieves ~97.8% F1 on both, but some edge cases remain.
- **LAME encoder only**: Training data was generated with LAME. Other MP3 encoders (FhG, Fraunhofer) may produce different spectral signatures.
- **44.1 kHz sample rate**: Files must be 44.1 kHz for accurate analysis. Other sample rates will be resampled by librosa.
- **Minimum file length**: Files shorter than ~50 seconds may have fewer sequences for aggregation, potentially reducing confidence.

## Research Foundation

- **SVM baseline**: D'Alessandro & Shi (2009), "MP3 Bit Rate Quality Detection through Frequency Spectrum Analysis"
- **CNN approach**: Informed by Hennequin et al. (2017, Deezer/ICASSP) and Seichter et al. (2016, Fraunhofer)
- **Global modulation**: Inspired by "Generalized Spoofing Detection Inspired from Audio Generation Artifacts" (2021)
- **VBR vs CBR discrimination at similar bitrates**: Novel contribution — no published work addresses this specific problem
