Metadata-Version: 2.4
Name: coding-cli-runtime
Version: 0.2.0
Summary: Reusable CLI runtime primitives for provider-backed automation workflows
Author-email: LLM Eval maintainers <llm-eval-maintainers@users.noreply.github.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/pj-ms/llm-eval/tree/main/packages/coding-cli-runtime
Project-URL: Repository, https://github.com/pj-ms/llm-eval
Project-URL: Issues, https://github.com/pj-ms/llm-eval/issues
Project-URL: Changelog, https://github.com/pj-ms/llm-eval/blob/main/packages/coding-cli-runtime/CHANGELOG.md
Keywords: cli,runtime,llm,automation,schema-validation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# coding-cli-runtime

[![PyPI](https://img.shields.io/pypi/v/coding-cli-runtime)](https://pypi.org/project/coding-cli-runtime/)
[![Python](https://img.shields.io/pypi/pyversions/coding-cli-runtime)](https://pypi.org/project/coding-cli-runtime/)
[![Build](https://github.com/pj-ms/llm-eval/actions/workflows/ci.yml/badge.svg)](https://github.com/pj-ms/llm-eval/actions/workflows/ci.yml)
[![License](https://img.shields.io/pypi/l/coding-cli-runtime)](LICENSE)

A Python library for orchestrating LLM coding agent CLIs — [Claude Code](https://docs.anthropic.com/en/docs/claude-code), [Codex](https://github.com/openai/codex), [Gemini CLI](https://github.com/google-gemini/gemini-cli), and [GitHub Copilot](https://docs.github.com/en/copilot).

These CLIs each have different invocation patterns, output formats, error
shapes, and timeout behaviors. This library normalizes all of that behind
a common `CliRunRequest` → `CliRunResult` contract, so your automation
code doesn't need provider-specific subprocess handling.

**What it does (and why not just `subprocess.run`):**

- Unified request/result types across all four CLIs
- Timeout enforcement with graceful process termination
- Provider-aware failure classification (retryable vs fatal)
- Built-in model catalog with defaults, reasoning levels, and capabilities
- Interactive session management for long-running generation tasks
- Zero runtime dependencies

## Installation

```bash
pip install coding-cli-runtime
```

Requires Python 3.10+.

## Examples

### Execute a provider CLI

```python
import asyncio
from pathlib import Path
from coding_cli_runtime import CliRunRequest, run_cli_command

request = CliRunRequest(
    cmd_parts=("codex", "--model", "o4-mini", "--quiet", "exec", "fix the tests"),
    cwd=Path("/tmp/my-project"),
    timeout_seconds=120,
)
result = asyncio.run(run_cli_command(request))

print(result.returncode)        # 0
print(result.error_code)        # "none"
print(result.duration_seconds)  # 14.2
print(result.stdout_text[:200])
```

Swap `codex` for `claude`, `gemini`, or `copilot` — the request/result
shape stays the same. A synchronous variant `run_cli_command_sync` is also
available.

### Pick a model from the provider catalog

```python
from coding_cli_runtime import get_provider_spec

codex = get_provider_spec("codex")
print(codex.default_model)   # "gpt-5.3-codex"
print(codex.model_source)    # "codex_cli_cache", "override", or "code"

for model in codex.models:
    print(f"  {model.name}: {model.description}")
```

The catalog covers all four providers — each with model names, reasoning
levels, default settings, and visibility flags.

Model lists are resolved with a three-tier fallback:

1. **User override** — drop a JSON file at
   `~/.config/coding-cli-runtime/providers/<provider>.json` to use your own
   model list immediately, without waiting for a package update.
2. **Live CLI cache** — for Codex, the library reads
   `~/.codex/models_cache.json` (auto-refreshed by the Codex CLI) when
   present. Other providers fall through because their CLIs don't expose a
   machine-readable model list.
3. **Hardcoded fallback** — the model list shipped with the package.

Override file format:

```json
{
  "default_model": "claude-sonnet-4-7",
  "models": [
    "claude-sonnet-4-7",
    {
      "name": "claude-opus-5",
      "description": "Latest opus model",
      "controls": [
        { "name": "effort", "kind": "choice", "choices": ["low", "high"], "default": "low" }
      ]
    }
  ]
}
```

Set `CODING_CLI_RUNTIME_CONFIG_DIR` to change the config directory
(default: `~/.config/coding-cli-runtime`).

### Decide whether to retry a failed run

```python
from coding_cli_runtime import classify_provider_failure

classification = classify_provider_failure(
    provider="gemini",
    stderr_text="429 Resource exhausted: rate limit exceeded",
)

if classification.retryable:
    print(f"Retryable ({classification.category}) — will retry")
else:
    print(f"Fatal ({classification.category}) — giving up")
```

Works for all four providers. Recognizes auth failures, rate limits,
network transients, and other provider-specific error patterns.

### Look up provider contract metadata

```python
from coding_cli_runtime import get_provider_contract, build_env_overlay, resolve_config_paths, render_prompt

# Get structured metadata for any supported provider
contract = get_provider_contract("claude")
print(contract.binary)                        # "claude"
print(contract.auth.api_key_env_var)          # "CLAUDE_API_KEY"
print(contract.paths.config_dir)              # "~/.claude"
print(contract.headless.approval.flag)        # "--dangerously-skip-permissions"

# Build env var overlay for subprocess
env = build_env_overlay(contract, api_key="sk-...", base_url="https://custom.example.com")
# {"CLAUDE_API_KEY": "sk-...", "ANTHROPIC_BASE_URL": "https://custom.example.com"}

# Resolve config paths for container mounts
host_dir, container_dir = resolve_config_paths(contract, containerized=True)
# ("/home/user/.claude", "/root/.claude")

# Resolve prompt delivery (stdin vs flag vs activation)
payload = render_prompt(contract.headless.prompt, "Fix the bug")
# payload.args = ()            (stdin delivery for Claude)
# payload.stdin_text = "Fix the bug"
```

`ProviderContract` is structured as nested sub-contracts
(`AuthContract`, `PathContract`, `HeadlessContract`) so consumers
can drill into whichever aspect they need. This is reference metadata,
not a command-construction control plane — consumers keep their own
command assembly and adopt contract fields selectively.

## Key types

| Type | Purpose |
|------|---------|
| `CliRunRequest` | Command spec: cmd, cwd, env, timeout, stream paths |
| `CliRunResult` | Result: returncode, stdout/stderr, duration, error code |
| `ErrorCode` | `none` · `spawn_failed` · `timed_out` · `non_zero_exit` |
| `ProviderSpec` | Provider catalog entry with models, controls, defaults |
| `ProviderContract` | Structured provider CLI metadata (auth, paths, headless launch) |
| `FailureClassification` | Classified error with retryable flag and category |

`run_interactive_session()` manages long-running CLI processes with
timeout enforcement, process-group cleanup, transcript mirroring, and
automatic retries. Only `cmd_parts`, `cwd`, `stdin_text`, and `logger` are
required — observability labels like `job_name` and `phase_tag` default to
sensible values so external callers don't need to invent them.

## Prerequisites

This package does **not** bundle any CLI binaries or credentials. You must
install and authenticate the relevant provider CLI yourself before using the
execution helpers.

## Status

Pre-1.0. API may change between minor versions.

## License

MIT
