Metadata-Version: 2.4
Name: cleanmonkey
Version: 0.1.0
Summary: One-call text cleanup: invisible characters, smart quotes, whitespace normalization.
Author-email: RexBytes <pythonic@rexbytes.com>
License: MIT
Project-URL: Homepage, https://github.com/RexBytes/cleanmonkey
Project-URL: Repository, https://github.com/RexBytes/cleanmonkey
Project-URL: Issues, https://github.com/RexBytes/cleanmonkey/issues
Keywords: text,cleanup,whitespace,unicode,normalize
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Text Processing
Classifier: Topic :: Text Processing :: Filters
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# cleanmonkey

One-call text cleanup for invisible characters, smart quotes, and whitespace normalization.

## Install

```bash
pip install cleanmonkey
```

## Quick Start

```python
from cleanmonkey import clean

# Sensible defaults handle the common garbage
clean("hello\u00a0world\u2019s \u2014 test")
# → "hello world's - test"

# Idempotent — safe to call twice
clean(clean(text)) == clean(text)
```

## What It Cleans (by default)

| Category | Examples | Result |
|---|---|---|
| Non-breaking spaces | `\u00a0`, `\u2007`, `\u202f` | Regular space |
| Zero-width chars | `\u200b`, `\u200c`, `\u200d`, `\ufeff` | Removed |
| Smart quotes | `\u2018` `\u2019` `\u201c` `\u201d` | `'` and `"` |
| Dashes | `\u2013` (en), `\u2014` (em) | `-` |
| Ellipsis | `\u2026` | `...` |
| Control chars | null, form feed, vertical tab | Removed |
| Line endings | `\r\n`, `\r` | `\n` |
| Multiple spaces | `"hello   world"` | `"hello world"` |
| Leading/trailing | `"  hello  "` | `"hello"` |

## Granular Control

Override any default:

```python
clean(text, smart_quotes=False)       # keep curly quotes
clean(text, dashes=False)             # keep em/en dashes
clean(text, fullwidth=True)           # also normalize fullwidth digits/letters
clean(text, collapse_spaces=False)    # keep multiple spaces
clean(text, strip=False)              # keep leading/trailing whitespace
```

## Profiles

```python
clean(text, profile="default")     # all normalizations (the default)
clean(text, profile="csv")         # default + fullwidth normalization
clean(text, profile="sql")         # default + fullwidth normalization
clean(text, profile="display")     # keep smart quotes & dashes; still clean invisible, control, whitespace, line endings
clean(text, profile="minimal")     # invisible chars only, no collapsing or stripping
clean(text, profile="aggressive")  # everything including fullwidth
```

## Batch Helpers

```python
from cleanmonkey import clean_column, clean_dict

# Clean a list (non-strings pass through)
clean_column(["hello\u00a0world", 42, None])
# → ["hello world", 42, None]

# Recursively clean dict values
clean_dict({"name": "John\u00a0Doe", "nested": {"val": "test\u200b"}})
# → {"name": "John Doe", "nested": {"val": "test"}}

# Also clean keys
clean_dict({"key\u00a0name": "val"}, keys=True)
# → {"key name": "val"}
```

## Inspect

Find out what's lurking in your text:

```python
from cleanmonkey import inspect

for info in inspect("hello\u00a0world\u200b"):
    print(f"{info.codepoint} {info.name} count={info.count} at {info.positions}")
# U+00A0 NO-BREAK SPACE count=1 at [5]
# U+200B ZERO WIDTH SPACE count=1 at [11]
```

## CLI

```bash
# Clean a file
cleanmonkey input.txt -o output.txt

# Pipe through stdin
cat dirty.csv | cleanmonkey > clean.csv

# Use a profile
cleanmonkey --profile csv input.txt

# Inspect mode — report what's in a file
cleanmonkey --inspect input.txt

# Machine-readable JSON inspect output
cleanmonkey --json input.txt

# Selective overrides
cleanmonkey --no-smart-quotes --fullwidth input.txt

# Preserve whitespace structure
cleanmonkey --no-strip --no-collapse-spaces input.txt

# Preserve line endings (CR/CRLF)
cleanmonkey --no-line-endings input.txt
```

## Built for LLMs

cleanmonkey is designed to work well as a tool for large language models. Invisible character cleanup is a constant source of silent bugs in LLM-driven data pipelines — non-breaking spaces break splits, zero-width characters corrupt comparisons, and smart quotes fail exact matches. Without cleanmonkey, LLMs end up generating repetitive `.replace()` chains that miss edge cases and waste tokens. A single `clean()` call handles all of it with a structured, idempotent result — no multi-step prompting or character-by-character debugging required. Fewer tokens in, clean data out.

## License

MIT
