# bicep-whatif-advisor

[![Tests](https://github.com/neilpeterson/bicep-whatif-advisor/actions/workflows/test.yml/badge.svg)](https://github.com/neilpeterson/bicep-whatif-advisor/actions/workflows/test.yml)
[![PyPI version](https://img.shields.io/pypi/v/bicep-whatif-advisor)](https://pypi.org/project/bicep-whatif-advisor/)
[![Python versions](https://img.shields.io/pypi/pyversions/bicep-whatif-advisor)](https://pypi.org/project/bicep-whatif-advisor/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

AI-powered deployment safety gate for Azure Bicep and ARM templates. Pipe Azure What-If output through an LLM (Anthropic Claude, Azure OpenAI, or Ollama) to detect infrastructure drift, validate PR intent alignment, and evaluate custom risk dimensions. Integrates into GitHub Actions and Azure DevOps with zero-config platform detection, automatic PR comments, and configurable risk thresholds. Also works as a standalone CLI for local What-If analysis.

## How It Works

Pipe Azure What-If output to the tool in your CI/CD pipeline. It auto-detects the platform, collects PR metadata and code diff, then sends everything to the LLM for analysis across two built-in risk categories:

- **Infrastructure Drift** - Detects changes not in your code (out-of-band modifications)
- **PR Intent Alignment** - Ensures changes match PR description

You can also add **custom risk assessment agents** via markdown files to evaluate additional dimensions like compliance, cost, risky operations, or naming conventions. See [Custom Agents](#custom-agents) below.

Known Azure What-If noise (etag, provisioningState, etc.) is filtered before LLM analysis. Results are posted as a PR comment and the pipeline exits with code 0 (safe) or 1 (unsafe) based on configurable thresholds.

**Example PR Comment:**

> ## production What-If Summary (non-blocking)
>
> | Risk Assessment | Risk Level | Key Concerns |
> |---|---|---|
> | Infrastructure Drift | High | Existing Paris Key Vault being modified when not present in code diff |
> | PR Intent Alignment | High | PR intends to add Berlin branch but What-If shows Paris branch changes |
>
> **Summary:** The What-If output shows modifications to existing Paris branch resources rather than creating new Berlin branch resources. Most changes appear to be template refactoring (hardcoded to dynamic references) and What-If noise (retention policies, metadata). The only significant change is disabling public network access on an existing Key Vault.
>
> <details><summary>📁 View changed resources (1 High Confidence)</summary>...</details>
>
> <details><summary>⚠️ Potential Azure What-If Noise (10 Low Confidence)</summary>...</details>
>
> <details><summary>📂 Raw What-If Output</summary>...</details>
>
> ### Verdict: ❌ UNSAFE
>
> **Reasoning:** This deployment is unsafe due to a complete mismatch between PR intent (adding Berlin branch) and What-If output (modifying existing Paris Key Vault). The changes indicate both infrastructure drift and unintended modifications to existing resources not mentioned in the PR description.
>
> ---
>
> *Generated by [bicep-whatif-advisor](https://github.com/neilpeterson/bicep-whatif-advisor)*

## Quick Start

**GitHub Actions:**
```yaml
- name: Deployment Safety Gate
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: |
    pip install bicep-whatif-advisor[anthropic]
    az deployment group what-if \
      --resource-group ${{ vars.AZURE_RESOURCE_GROUP }} \
      --template-file main.bicep \
      --exclude-change-types NoChange Ignore \
      | bicep-whatif-advisor
```

**Azure DevOps:**
```yaml
- script: |
    pip install bicep-whatif-advisor[anthropic]
    az deployment group what-if \
      --resource-group $(RESOURCE_GROUP) \
      --template-file main.bicep \
      --exclude-change-types NoChange Ignore \
      | bicep-whatif-advisor
  env:
    ANTHROPIC_API_KEY: $(ANTHROPIC_API_KEY)
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)
```

## Configuration

### Risk Thresholds

Each risk bucket has an independent threshold: `low`, `medium`, `high` (default: `high` — most permissive).

```bash
# Block on medium or high risk
bicep-whatif-advisor \
  --drift-threshold medium \
  --intent-threshold medium
```

Skip individual buckets with `--skip-drift` or `--skip-intent`.

### Custom Agents

Add custom risk assessment dimensions using markdown files with YAML frontmatter. Each file defines a new risk bucket that runs alongside the built-in ones:

```markdown
---
id: compliance
display_name: Compliance Review
default_threshold: high
---

**Compliance Risk:**
Evaluate whether changes comply with organizational policies.

Risk levels for compliance:
- high: Encryption disabled, public access enabled, data residency violations
- medium: Policy assignment changes, unapproved resource types
- low: Tag updates for compliance metadata, monitoring additions
```

Point the tool at a directory of agent files:

```bash
bicep-whatif-advisor --ci \
  --agents-dir ./agents/ \
  --agent-threshold compliance=medium
```

Custom agents support `--agent-threshold` for per-agent thresholds and `--skip-agent` to disable individual agents. See the [User Guide](docs/guides/USER_GUIDE.md#custom-agents) for details.

### LLM Providers

Defaults to Anthropic Claude. Also supports Azure OpenAI and Ollama:

```bash
# Azure OpenAI
pip install bicep-whatif-advisor[azure]
bicep-whatif-advisor --provider azure-openai

# Local Ollama
pip install bicep-whatif-advisor[ollama]
bicep-whatif-advisor --provider ollama --model llama3.1
```

### Config File

Define settings in a YAML file instead of passing many CLI flags. CLI flags always override config values:

```yaml
# whatif-config.yaml
provider: azure-openai
ci: true
drift_threshold: medium
agents_dir: ./agents
```

```bash
bicep-whatif-advisor --config-file whatif-config.yaml
```

### Additional Options

```bash
# Output formats: table (default), json, markdown
bicep-whatif-advisor --format json

# Include raw What-If output in PR comment
bicep-whatif-advisor --include-whatif

# Label comments for multi-environment pipelines
bicep-whatif-advisor --comment-title "Production"

# Non-blocking mode (report without failing pipeline)
bicep-whatif-advisor --no-block
```

See the **[User Guide](docs/guides/USER_GUIDE.md)** for all CLI flags and the **[CI/CD Integration Guide](docs/guides/CICD_INTEGRATION.md)** for complete pipeline setup.

## Documentation

- [Quick Start](docs/guides/QUICKSTART.md) - Get running in 5 minutes
- [User Guide](docs/guides/USER_GUIDE.md) - Complete feature reference and CLI flags
- [CI/CD Integration](docs/guides/CICD_INTEGRATION.md) - Pipeline setup for GitHub Actions, Azure DevOps, etc.
- [Risk Assessment](docs/guides/RISK_ASSESSMENT.md) - Deep dive into AI risk evaluation
- [Technical Specifications](docs/specs/) - Architecture, design, and module specs

## Support

- **Issues**: Report bugs or request features via repository issues
- **Contributing**: Pull requests welcome!
- **License**: MIT - see [LICENSE](LICENSE) for details
