# Skill: Enable LangSmith Observability

Use when you want to trace pipeline executions — see per-agent latency, LLM inputs/outputs,
tool call arguments, and costs in a web UI.

## What LangSmith Shows

- **Project view**: all runs for a pipeline, with status and duration
- **Run detail**: each node (agent) as a nested span with full input/output
- **LLM call detail**: the exact messages sent to the model, the response, and token counts
- **Tool calls**: arguments passed to each tool and the tool's return value
- **Cost estimate**: estimated API cost per run
- **Error traces**: full stack traces for failed runs

## Setup

### Step 1: Get a LangSmith API Key

1. Sign up at https://smith.langchain.com (free tier available)
2. Go to Settings → API Keys → Create API Key
3. Copy the key (format: `ls__...`)

### Step 2: Add to `.env`

```bash
# .env
LANGSMITH_API_KEY=ls__...
LANGSMITH_PROJECT=my-pipeline-dev     # project name in LangSmith UI
LANGSMITH_TRACING=true
```

That is all — the framework activates LangSmith automatically when these are set.

## How It Is Initialized

LangSmith is configured in two places:

**`arccrew/config.py` — `_configure_logging()`**:
```python
if settings.langsmith_tracing and settings.langsmith_api_key:
    import os
    os.environ["LANGCHAIN_TRACING_V2"] = "true"
    os.environ["LANGCHAIN_API_KEY"] = settings.langsmith_api_key
    if settings.langsmith_project:
        os.environ["LANGCHAIN_PROJECT"] = settings.langsmith_project
```

**`arccrew/llm.py` — `_configure_langsmith()`**:
```python
def _configure_langsmith() -> None:
    settings = get_settings()
    if settings.langsmith_tracing and settings.langsmith_api_key:
        os.environ["LANGCHAIN_TRACING_V2"] = "true"
        os.environ["LANGCHAIN_API_KEY"] = settings.langsmith_api_key
        if settings.langsmith_project:
            os.environ["LANGCHAIN_PROJECT"] = settings.langsmith_project
        logger.info(f"LangSmith tracing enabled → project: {settings.langsmith_project or 'default'}")
```

LangChain's tracing picks up `LANGCHAIN_TRACING_V2=true` and sends traces automatically
to LangSmith — no code changes needed in your agents or tools.

## Verifying It Works

Start the server and call your pipeline:

```bash
arccrew serve
curl -X POST http://localhost:8000/api/run \
  -H "Content-Type: application/json" \
  -d '{"pipeline": "my_pipeline", "tasks": [{"description": "test"}]}'
```

You should see in the logs:
```
LangSmith tracing enabled → project: my-pipeline-dev
```

Then go to https://smith.langchain.com → select your project → you should see the run.

If no runs appear:
1. Check `LANGSMITH_TRACING=true` (not `True` or `1` — must be lowercase `true`)
2. Check `LANGSMITH_API_KEY` starts with `ls__`
3. Check `LANGSMITH_PROJECT` matches the project name in the UI (it auto-creates on first run)
4. Run `arccrew check` to verify settings are loaded

## Enterprise Alternative

If your client cannot send data to LangSmith (compliance, PII, air-gapped), use OpenTelemetry instead — see `/enable-otel`.

## Using Traces to Compare Models

LangSmith makes it easy to compare model performance:

1. Run your pipeline with `AGENT_MODEL=anthropic/claude-haiku-4-5-20251001`
2. Run the same pipeline with `AGENT_MODEL=anthropic/claude-sonnet-4-20250514`
3. In LangSmith, compare the two runs side-by-side:
   - Token counts and cost
   - Response quality (look at LLM outputs)
   - Total duration

Use the same `LANGSMITH_PROJECT` for both runs, then filter by date or run metadata.

To tag runs for easier comparison, set the project name per run:

```bash
LANGSMITH_PROJECT=haiku-test arccrew serve
LANGSMITH_PROJECT=sonnet-test arccrew serve
```

## Filtering Traces

In the LangSmith UI:
- **By project**: use the project dropdown to isolate runs for a specific pipeline
- **By status**: filter to "Error" to find all failed runs
- **By date**: use the date range filter for a specific test session
- **By metadata**: add `metadata` to your run via LangChain callbacks (advanced)

## Cost: Free Tier

LangSmith's free tier (as of early 2026) includes:
- Unlimited projects
- Several thousand traces per month
- 14-day retention

For development and testing, the free tier is more than sufficient. Upgrade only if you
have high-volume production tracing needs.

## Disabling Tracing

To disable tracing without removing the key from `.env`:

```bash
# .env
LANGSMITH_TRACING=false
```

Or remove the three LangSmith lines entirely. When `LANGSMITH_TRACING` is `false` or
the key is not set, the `_configure_langsmith()` function logs nothing and tracing is off.

## Privacy Consideration

LangSmith sends LLM inputs, outputs, and tool call data to Anthropic/LangChain's cloud.
Do not enable in production environments where prompts contain PII or confidential data
unless you have reviewed LangSmith's data retention and privacy policies.

For local-only tracing, run LangSmith self-hosted or use alternative open-source options
like Phoenix (Arize) or Langfuse.
