Metadata-Version: 2.2
Name: agently-devtools
Version: 0.1.2
Summary: Observation and developer tooling companion package for Agently
Author-email: Agently Team <developer@agently.tech>
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: agently<4.2.0,>=4.1.0.2
Requires-Dist: fastapi<1.0,>=0.104
Requires-Dist: httpx<0.29.0,>=0.28.1
Requires-Dist: uvicorn<1.0,>=0.30
Provides-Extra: dev
Requires-Dist: anyio<5,>=4; extra == "dev"
Requires-Dist: build<2.0,>=1.2; extra == "dev"
Requires-Dist: pytest<9.0.0,>=8.4.0; extra == "dev"
Requires-Dist: pytest-asyncio<2.0.0,>=1.0.0; extra == "dev"

# agently-devtools

`agently-devtools` is the local observation, evaluation, and playground companion package for `agently`.

## Install

```bash
pip install -U agently agently-devtools
```

Compatible with `agently >=4.1.0.2,<4.2.0`.

Python `>=3.10`

## Start

```bash
agently-devtools start
```

Default local address:

- Console: `http://127.0.0.1:15596/`
- Ingest: `http://127.0.0.1:15596/observation/ingest`

Explicit listener options:

```bash
agently-devtools start --host 0.0.0.0 --port 15596 --path /observation/ingest --no-open
```

## Observation Bridge

```python
from agently import Agently
from agently_devtools import ObservationBridge

bridge = ObservationBridge(
    app_id="your_app_id",
    group_id="your_group_id",
)
bridge.register(Agently)
```

Explicit endpoint styles are also supported:

```python
ObservationBridge(host="127.0.0.1", port=15596, path="/observation/ingest")
ObservationBridge("https://devtools.example.com/observation/ingest")
```

Use selective watch when you do not want to upload every runtime event:

```python
bridge = ObservationBridge(
    app_id="your_app_id",
    auto_watch=False,
)
bridge.watch(agent, flow, lookup_reference)
bridge.register(Agently)
```

TriggerFlow executions automatically publish their flow definition, so the execution graph can show the full static structure, including branches that have not been hit yet.

## Scenario Evaluations

```python
from agently_devtools import EvaluationBridge, EvaluationCase, EvaluationRunner

bridge = EvaluationBridge(
    base_url="http://127.0.0.1:15596",
    app_id="your_app_id",
    group_id="your_eval_group",
)
runner = EvaluationRunner(bridge=bridge)

binding = bridge.bind_agent_factory(
    lambda: build_agent(),
    suite_id="support-suite",
    target_name="support-agent",
)

report = runner.run(
    binding,
    cases=[
        EvaluationCase(case_id="refund", input="Need a refund for a duplicate payment."),
        EvaluationCase(case_id="shipping", input="Where is my shipment now?"),
    ],
    rules=[
        lambda record: bool(record.output),
        lambda record: record.error is None,
    ],
    rounds=2,
)
```

Use the binding helper that matches your target:

- `bind_agent(...)` / `bind_agent_factory(...)`
- `bind_request(...)` / `bind_request_factory(...)`
- `bind_triggerflow(...)` / `bind_triggerflow_factory(...)`

`EvaluationBridge` already records rounds into DevTools and keeps observed run linkage automatically. You do not need an extra `ObservationBridge` just for Scenario Evaluations.
