Metadata-Version: 2.4
Name: chat-template-detector
Version: 0.1.0
Summary: Detect chat template mismatches in LLM fine-tuning
Author: ClarifyIntel
License: MIT
Project-URL: Homepage, https://github.com/clarifyintel/chat-template-detector
Project-URL: Repository, https://github.com/clarifyintel/chat-template-detector
Project-URL: Issues, https://github.com/clarifyintel/chat-template-detector/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.0
Requires-Dist: jinja2>=3.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: black>=22.0; extra == "dev"
Requires-Dist: flake8>=5.0; extra == "dev"
Dynamic: license-file

# Chat Template Detector

Detect chat template mismatches between training and inference in LLM fine-tuning.

## The Problem

You fine-tune an LLM with one chat template format. You run inference with a different format. The model outputs garbage. No error message. Hours wasted debugging.

This tool catches template mismatches before they waste your time and GPU credits.

## Installation

```bash
pip install chat-template-detector
```

## Usage

```bash
# Validate training data against inference config
chat-template-detector validate \
  --training-file train.jsonl \
  --inference-config config.yaml

# Validate against a specific model
chat-template-detector validate \
  --training-file train.jsonl \
  --model meta-llama/Llama-2-7b-chat-hf

# Check a single formatted text file
chat-template-detector check samples/sample_chatml.txt
```

## Common Issues Detected

- ChatML vs Llama format mismatch
- Missing special tokens
- Incorrect token ordering
- BOS/EOS token mismatches
- Role name inconsistencies

## Examples

Training file uses ChatML:
```json
{"messages": [{"role": "user", "content": "Hello"}, {"role": "assistant", "content": "Hi"}]}
```

Inference expects Llama format:
```
<s>[INST] Hello [/INST] Hi</s>
```

Detector catches this mismatch and shows exactly what's wrong.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md)

## License

MIT License - see [LICENSE](LICENSE) for details

## Author

Built by [ClarifyIntel](https://clarifyintel.com) - We clarify complex tech. Kubernetes, AI/ML, DevOps, Security — we write, we teach, we solve.
