Metadata-Version: 2.4
Name: agent-lint-cli
Version: 0.3.0
Summary: Quality and security platform for AI agents. Validate MCP servers, scan for vulnerabilities, ship reliable agents.
Project-URL: Homepage, https://github.com/JDoornink/agent-lint
Project-URL: Issues, https://github.com/JDoornink/agent-lint/issues
Author: agent-lint contributors
License-Expression: MIT
Keywords: agents,ai,lint,mcp,security,testing,validation
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Pytest
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: click>=8.0
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.22; extra == 'dev'
Provides-Extra: testing
Requires-Dist: pytest-asyncio>=0.23; extra == 'testing'
Requires-Dist: pytest>=8.0; extra == 'testing'
Description-Content-Type: text/markdown

# agent-lint

> ESLint for agents.

Validate MCP servers, scan for vulnerabilities, write integration tests, and ship reliable agents.

[![PyPI](https://img.shields.io/pypi/v/agent-lint-cli)](https://pypi.org/project/agent-lint-cli/)
[![Python](https://img.shields.io/pypi/pyversions/agent-lint-cli)](https://pypi.org/project/agent-lint-cli/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

---

## Install

```bash
pip install agent-lint-cli
```

---

## Validate an MCP Server

```bash
agent-lint validate https://my-mcp-server.com
```

```
🔍 Validating MCP Server: https://my-mcp-server.com

Schema Validation
  ✓ Valid JSON-RPC 2.0 endpoint
  ✓ Tools list returned (5 tools)
  ✓ Resources endpoint responds

Security Analysis
  ⚠ [MEDIUM] Tool "run_query" has parameter "sql" without type validation
  ⚠ [MEDIUM] Tool "execute_task" has risky name pattern
  ✓ No credential patterns detected
  ✓ Rate limiting headers present

Tool Quality
  ✓ All tools have descriptions
  ✗ Tool "helper" missing parameter descriptions

Performance
  ✓ Response time: 245ms
  ⚠ Large response payload: 2.3MB

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Security Score: 75/100
Quality Score:  68/100
```

### Options

| Flag | Description |
|------|-------------|
| `--format console\|json\|sarif` | Output format (default: `console`) |
| `--fail-under N` | Exit 1 if overall score is below N (0–100) |
| `--fail-on-security` | Exit 1 if any security issues are found |
| `--security-level strict\|standard\|permissive\|none` | Check strictness (default: `standard`) |
| `--dynamic` | Run dynamic security tests (calls tools with payloads) |
| `--policy PATH` | Path to `.agent-lint.yaml` policy file |
| `--junit-xml PATH` | Write JUnit XML results to file |

---

## CI/CD Integration

### GitHub Actions

```yaml
# .github/workflows/mcp-security.yml
name: MCP Security

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ./.github/actions/agent-lint
        with:
          url: https://my-mcp-server.com
          fail-on-security: "true"
          security-level: strict
```

The action automatically uploads SARIF results to the GitHub Security tab.

### Inline with CLI

```bash
# Fail on any security issue
agent-lint validate https://my-server.com --fail-on-security

# Require minimum score
agent-lint validate https://my-server.com --fail-under 80

# SARIF output for GitHub Security tab
agent-lint validate https://my-server.com -f sarif > results.sarif

# JUnit XML for test reporting
agent-lint validate https://my-server.com --junit-xml results.xml

# Dynamic security testing (calls tools with injection payloads)
agent-lint validate https://my-server.com --dynamic
```

---

## Security Policies

Create an `.agent-lint.yaml` file to enforce rules across your team:

```yaml
# .agent-lint.yaml
security:
  level: strict

  rules:
    dangerous_patterns: error
    missing_validation: error
    overly_permissive: warn
    credential_exposure: error

  exceptions:
    - tool: admin_execute
      reason: "Admin-only endpoint with auth middleware"
      rules:
        - dangerous_patterns
```

Then run with:

```bash
agent-lint validate https://my-server.com --policy .agent-lint.yaml
```

Policy security level overrides `--security-level` unless you pass `--security-level` explicitly.

---

## MCP Testing (pytest)

agent-lint ships a pytest plugin and fixtures for writing integration tests against MCP servers.

### Install testing extras

```bash
pip install "agent-lint-cli[testing]"
```

### Write tests

```python
# tests/test_my_server.py
from agent_lint import mcp


class TestQueryTool:

    async def test_basic_query(self, mcp_client):
        response = await mcp_client.call_tool("query_database", {"sql": "SELECT 1"})
        mcp.assert_valid_response(response)

    async def test_rejects_sql_injection(self, mcp_client):
        response = await mcp_client.call_tool(
            "query_database",
            {"sql": "'; DROP TABLE users; --"},
        )
        mcp.assert_error(response)

    async def test_response_time(self, mcp_client):
        response = await mcp_client.call_tool("query_database", {"sql": "SELECT 1"})
        mcp.assert_response_time(response, max_ms=500)

    async def test_no_secrets_in_response(self, mcp_client):
        response = await mcp_client.call_tool("get_config", {})
        mcp.assert_no_secrets(response)
```

### Run against a real server

```bash
# Against a live server
agent-lint test tests/ --mcp-url https://my-mcp-server.com

# Or with pytest directly
pytest tests/ --mcp-url https://my-mcp-server.com
```

### Run with the mock server (no real server needed)

```python
# tests/test_mock.py
async def test_with_mock(mock_mcp_server, mcp_client):
    mock_mcp_server.add_tool(
        "get_weather",
        description="Get weather for a city",
        input_schema={
            "type": "object",
            "properties": {"city": {"type": "string"}},
            "required": ["city"],
        },
        handler=lambda args: {"content": [{"type": "text", "text": f"Sunny in {args['city']}"}]},
    )

    response = await mcp_client.call_tool("get_weather", {"city": "London"})
    mcp.assert_valid_response(response)
    mcp.assert_content_matches(response, r"Sunny")
```

When `--mcp-url` is not passed, `mcp_client` automatically uses the mock server.

---

## Assertion Reference

| Assertion | Description |
|-----------|-------------|
| `mcp.assert_valid_response(r)` | Response has a result and no error |
| `mcp.assert_error(r, code=None)` | Response is an error (optionally assert error code) |
| `mcp.assert_result_contains(r, key, value)` | Result dict contains key (optionally assert value) |
| `mcp.assert_content_text(r, expected)` | Response has a text content block matching string |
| `mcp.assert_content_matches(r, pattern)` | Any text content block matches regex |
| `mcp.assert_tool_exists(r, name)` | `tools/list` response contains a named tool |
| `mcp.assert_no_secrets(r)` | Response does not contain credential patterns |
| `mcp.assert_response_time(r, max_ms)` | Response was returned within the time limit |

---

## Security Checks

agent-lint runs these checks by default (static analysis on tool definitions):

| Check | Severity | Description |
|-------|----------|-------------|
| Dangerous tool names | Medium | Names like `execute`, `shell`, `eval`, `run_command` |
| Risky parameters without validation | Medium | `sql`, `cmd`, `path`, `url` without JSON schema type |
| Overly permissive descriptions | Medium | Descriptions claiming "any file", "any command" |
| Credential exposure | High | API keys, tokens, secrets in responses |
| Missing rate limiting | Low | No rate-limit headers on responses |

With `--dynamic`, agent-lint also calls tools with injection payloads to detect runtime vulnerabilities:

| Payload Class | Example Parameters Targeted |
|---------------|-----------------------------|
| SQL injection | `sql`, `query`, `filter` |
| Path traversal | `path`, `file`, `filename` |
| Command injection | `cmd`, `command`, `exec`, `shell` |
| XSS | `html`, `content`, `body` |
| SSRF | `url`, `uri`, `endpoint` |

> **Note:** Static analysis is a first line of defense on tool definitions only. It cannot analyze handler code or detect all runtime vulnerabilities. Use `--dynamic` for deeper testing.

---

## Validate an A2A Agent

```bash
agent-lint validate https://my-agent.com --protocol a2a
```

```
🔍 Validating A2A Agent: https://my-agent.com

Schema Validation
  ✓ Agent card fetched (/.well-known/agent.json)
  ✓ Required fields present (name, url, version, skills)
  ✓ All skills have IDs

Security Analysis
  ⚠ [HIGH] Agent served over HTTP — use HTTPS in production
  ✓ Authentication schemes defined
  ✓ No injection patterns in skill descriptions

Quality
  ✓ Agent description present
  ✓ All skills have descriptions and tags

Performance
  ✓ Response time: 87ms
  ✓ Payload size: 1.2KB

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Security Score: 80/100
Quality Score:  100/100
```

### A2A Security Checks

| Check | Severity | Description |
|-------|----------|-------------|
| HTTPS enforcement | High | Agent URL must use HTTPS |
| Missing authentication | High | No `authentication.schemes` defined |
| Overly broad skills | Medium | Descriptions claiming "any file", "any command" |
| Prompt injection risk | Critical | Injection patterns in skill descriptions |
| Missing input schema | Low | Skills with no `inputModes`/`outputModes` |
| Missing version | Low | No version or placeholder version (0.0.0) |

---

## A2A Testing (pytest)

agent-lint includes a pytest plugin and fixtures for writing integration tests against A2A agents.

### Write tests

```python
# tests/test_my_agent.py
from agent_lint import a2a


class TestMyAgent:

    async def test_card_is_valid(self, a2a_client):
        card = await a2a_client.get_card()
        a2a.assert_valid_card(card)
        a2a.assert_skill_exists(card, "weather")

    async def test_skill_completes(self, a2a_client):
        response = await a2a_client.send_message("What is the weather?", skill_id="weather")
        a2a.assert_task_completed(response)
        a2a.assert_artifact_matches(response, r"sunny|cloudy|rain")

    async def test_no_secrets_in_response(self, a2a_client):
        response = await a2a_client.send_message("Hello", skill_id="weather")
        a2a.assert_no_secrets(response)
```

### Run against a real agent

```bash
# Against a live agent
agent-lint test tests/ --a2a-url https://my-agent.com

# Or with pytest directly
pytest tests/ --a2a-url https://my-agent.com
```

### Run with the mock agent (no real agent needed)

```python
# tests/test_mock_agent.py
from agent_lint import a2a


async def test_with_mock(mock_a2a_agent, a2a_client):
    mock_a2a_agent.set_card(name="Weather Agent", description="Provides weather forecasts")
    mock_a2a_agent.add_skill(
        "weather",
        description="Get current weather for a location",
        tags=["weather", "forecast"],
        handler=lambda text, history: {"type": "text", "text": "Sunny, 72°F"},
    )

    card = await a2a_client.get_card()
    a2a.assert_valid_card(card)

    response = await a2a_client.send_message("London weather", skill_id="weather")
    a2a.assert_task_completed(response)
    a2a.assert_artifact_matches(response, r"Sunny")
```

When `--a2a-url` is not passed, `a2a_client` automatically uses the mock agent.

### Test MCP and A2A together

Both `mcp_client` and `a2a_client` fixtures are always available. Use them in the same test to validate workflows that span both protocols:

```python
async def test_combined_workflow(mcp_client, a2a_client):
    # Fetch data via MCP tool
    data_response = await mcp_client.call_tool("get_data", {"id": "123"})
    mcp.assert_valid_response(data_response)

    # Send it to an A2A agent for analysis
    task = await a2a_client.send_message(str(data_response.result), skill_id="analyze")
    a2a.assert_task_completed(task)
```

---

## A2A Assertion Reference

| Assertion | Description |
|-----------|-------------|
| `a2a.assert_valid_card(r)` | Card was fetched and has required fields |
| `a2a.assert_skill_exists(r, skill_id)` | Agent card contains the named skill |
| `a2a.assert_task_completed(r)` | Task reached the `completed` state |
| `a2a.assert_task_failed(r)` | Task reached the `failed` state |
| `a2a.assert_task_state(r, state)` | Task is in the given state |
| `a2a.assert_artifact_contains(r, text)` | Any artifact text part contains exact text |
| `a2a.assert_artifact_matches(r, pattern)` | Any artifact text part matches regex |
| `a2a.assert_no_secrets(r)` | Response does not contain credential patterns |
| `a2a.assert_response_time(r, max_ms)` | Response was returned within the time limit |

---

## Roadmap

| Version | Status | Features |
|---------|--------|----------|
| v0.1 | Released | MCP validation + basic security |
| v0.2 | Released | MCP testing, pytest plugin, policy engine, CI/CD |
| **v0.3** | **Current** | A2A validation and testing |
| v1.0 | Planned | Observability, trace capture and replay |
