Metadata-Version: 2.4
Name: clab-integrator
Version: 0.1.0
Summary: Containerlab EDA Integrator — parse XLS, run clab, and build EDA resources
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: jinja2>=3.0.0
Requires-Dist: paramiko>=3.0.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: pyyaml>=6.0.0
Requires-Dist: rich>=13.0.0
Requires-Dist: urllib3>=2.0.0
Requires-Dist: openpyxl>=3.0.0
Requires-Dist: pandas>=2.0.0

# Containerlab EDA Integrator Tool

This package extends the [clab-eda-connector](https://github.com/eda-labs/clab-connector) tool by spinning up both Containerlab and EDA using one of 4 XLS templates as input. In addition to initializing Containerlab topologies and connecting them to EDA, this script also 1) upload overlay, underlay, allocation pools, index pools, and node management CRs to a new EDA namespace, 2) configures TopoNode productionAddresses for node sync, and 3) automatically creates a subnet and IP pool for MSP/shared clusters. Settings such as software version and yang schema can be configured by the user.

Integrate your [Containerlab](https://containerlab.dev/) topology seamlessly with [EDA (Event-Driven Automation)](https://docs.eda.dev)

## 🚨 Important Requirements

- **Linux VM and IPSec tunnel:** A Linux VM with an IPSec tunnel to your EDA cluster. Run the tool from this VM (download the script or clone the repo here).

- **Other:** Reference [clab-eda-connector](https://github.com/eda-labs/clab-connector) for any other requirements, as this script is an extension of clab-eda-connector.

## Installation

This project is managed with [uv](https://docs.astral.sh/uv/). Install uv if needed: `curl -LsSf https://astral.sh/uv/install.sh | sh`.

Install the CLI from this Git repo:

```bash
uv tool install 'git+ssh://git@gitlabe1.ext.net.nokia.com/edasaas/coopspring2026.git'
```

Use `...@branch` or `...@v0.1.0` in the Git URL to pin a branch or tag. For HTTPS: `uv tool install 'git+https://gitlabe1.ext.net.nokia.com/edasaas/coopspring2026.git'`.

**From PyPI** (once the package is published):

```bash
uv tool install clab-integrator
```

### Publishing to PyPI (for maintainers)

To make `uv tool install clab-integrator` work for everyone, publish the package to [PyPI](https://pypi.org/):

1. **Create a PyPI account** at [pypi.org](https://pypi.org/account/register/) (and [test.pypi.org](https://test.pypi.org/) for testing).
2. **Configure auth** (one of):
   - **Trusted publishing** (recommended): [Configure GitHub Actions](https://docs.pypi.org/trusted-publishers/) so releases publish automatically.
   - **API token**: Create a token on PyPI, then `uv publish --token pypi-...`.
   - **Keyring**: Store credentials with `keyring set https://upload.pypi.org/legacy/ __token__` (use your PyPI API token as the password).
3. **Build and publish** from the repo root:
   ```bash
   uv build
   uv publish
   ```
   For a dry run or Test PyPI first: `uv publish --repository testpypi`.
4. **Bump version** in `pyproject.toml` for each release, then run `uv build` and `uv publish` again.

After the first publish, anyone can install with `uv tool install clab-integrator`.

## Setup: config, credentials, and XLS templates (required)

**Recommended flow:** (1) Download the XLS templates and add `config.yaml` and `.env` to a working directory. (2) Edit fields in the templates and config as desired—**the tables in the XLS must keep their shape** (do not add/remove columns or change structure). (3) Run the tool with **`start`**, providing your XLS file as input (via prompt, `-x`/`--xls`, or `XLS_PATH` in `.env`).

### Flow

1. **Create a working directory** (e.g. your project or lab folder).
2. **Download the XLS templates** and place them in that directory (or a path you will pass to the tool).
3. **Add `config.yaml`** — Copy the [config.yaml](#configyaml) example below (or from the repo) into that directory. Adjust non-sensitive settings (software_version, log_level, stage_dir, etc.). Cluster type is inferred from `.env`: if `EDA_SAAS_ORG_NAME` is set the script treats the cluster as shared/MSP; if not set it assumes dedicated and uses Keycloak auth.
4. **Add `.env`** — Copy `.env.example` to `.env` in the same directory. Fill in EDA URL, user, password, namespace, and any optional Keycloak/SaaS variables. Do not commit `.env`.
5. **Edit the XLS and config as needed** — Change field values to match your topology and environment. **Keep the shape of the tables in the XLS unchanged** (same columns and structure; only edit cell values).
6. **Run the tool** from that directory so it finds `config.yaml` and `.env`:
   ```bash
   cd /path/to/your/dir
   clab-integrator
   ```
   At the prompt, run **`start`** and provide the path to your edited XLS file when asked (or pass it via `-x`/`--xls` or `XLS_PATH` in `.env`).
   Or pass the files explicitly from anywhere:
   ```bash
   clab-integrator -c /path/to/your/dir/config.yaml -e /path/to/your/dir/.env -x path/to/your-topology.xlsx
   ```

To change settings or credentials, edit `config.yaml` and `.env` (and XLS fields if needed); keep XLS table shape unchanged. Then run the tool again.

## Configuration reference

### `.env.example`

Copy to `.env` and fill in values. Do not commit `.env`.

```bash
# EDA Integrator — copy to .env and fill in secrets (do not commit .env)
# Non-sensitive options (log_level, log_dir, stage_dir) live in config.yaml.
# Cluster type: if EDA_SAAS_ORG_NAME is set → shared/MSP; if not set → dedicated (Keycloak auth).

# --- Required for EDA login ---
EDA_URL=https://manage.example.eda.nokia.com
EDA_USER=your-eda-username
EDA_PASSWORD=your-eda-password

# --- Namespace and license ---
DEFAULT_NAMESPACE=your-namespace
# SRL_LICENSE=your-srl-license-key

# --- Optional: Keycloak (if Keycloak admin API is used) ---
# KC_SECRET=your-kc-client-secret
# KC_USER=admin
# KC_PASSWORD=admin

# --- EDA SaaS org: if set, script treats cluster as shared/MSP; if not set, assumes dedicated ---
# EDA_SAAS_ORG_NAME=your-org-name

# --- Optional: Excel topology path (can also pass via -x/--xls or prompt at runtime) ---
# XLS_PATH=path/to/topology.xlsx
```

### config.yaml

Non-sensitive app config. Do not put secrets here; use `.env` for credentials.

```yaml
# Non-sensitive app config. Do not put secrets here; use .env for credentials.

# SRL / node profile version (e.g. 24.10.1, 25.7.2); used for images and EDA schemaprofiles
software_version: "25.7.2"

# SRL YANG schemaprofile zip build number (e.g. 492, 266); used in EDA yang_path:
# .../schemaprofiles/srlinux-ghcr-{software_version}/srlinux-{software_version}-{srl_yang_schema_build}.zip
srl_yang_schema_build: 266

# Logging
log_level: "DEBUG"
# Path to log file (default log output)
log_dir: "out/logs/eda-integrator.log"

# Stage directory for parse/preprocess output (warning: overwritten by pipeline)
stage_dir: "out/stage"

# Cluster type is inferred from .env: if EDA_SAAS_ORG_NAME is set → shared/MSP (SaaS auth);
# if not set → script assumes dedicated and uses Keycloak (KC_SECRET or KC_USER+KC_PASSWORD).
```

### Cluster type and authentication

**Cluster type is inferred from `.env` (there is no `cluster_model` in config):**

- **If `EDA_SAAS_ORG_NAME` is set** — The script treats the cluster as **shared or MSP**. It uses EDA SaaS authentication (organization-scoped). Use this for shared/MSP deployments.
- **If `EDA_SAAS_ORG_NAME` is not set** — The script **assumes the cluster is dedicated** and uses a different authentication method: either **Keycloak with client secret** (`KC_SECRET`) or **Keycloak admin flow** (`KC_USER` and `KC_PASSWORD`). Set one of these in `.env` for dedicated clusters.

You must have at least one auth method configured: `EDA_SAAS_ORG_NAME` for shared/MSP, or `KC_SECRET` or `KC_USER`+`KC_PASSWORD` for dedicated.

## Flow: how the tool works

### Step 1 – Parsing and templating

- The tool **parses input XLS (Excel) files** and converts them into dictionaries.
- These dictionaries are passed into **Jinja2 (J2) template files** to produce **EDA-compatible YAML**.
- You provide the **path to the XLS sheet** via `-x`/`--xls`, `XLS_PATH` in `.env`, or when prompted at start/parse.
- **Output of this step** (written to the stage directory) includes parameters for:
  - Toponodes, interfaces, topalinks  
  - Underlay (fabric) and overlay (virtual network)  
  - System IPs, filters, policies, ASNs, IBGP/eBGP  

### Step 2 – Containerlab deployment and staging

- **Containerlab** uses a generated **`topology.yaml`** (from the XLS parsing step) to **spin up nodes** according to the template.
- **After deployment**, management IPs from the running Clab environment are **parsed and written into the toponode YAMLs** under the **`productionAddress`** field for later sync with EDA.
- **Other generated files** (e.g. index pool parameters) are **staged in the same output (stage) directory** as the parsed topology and toponode files.

### Step 3 – EDA integration

- **Namespaces** are auto-created on EDA, along with **node management CRs**.
- **Clab nodes and interfaces** are auto-created on EDA.
- **Overlay and underlay** configuration is pushed to EDA, along with **index pools, IP pools, and subnet pools**.

## CLI commands

The interactive CLI runs the full pipeline or individual steps.

**Main commands:**

- **`start`** (or **`deploy`**) – Run the full pipeline: parse → preprocess → upload (prompts for XLS path if not set).
- **`inspect`** – Run Containerlab inspect for the stage topology and optionally ping an EDA IP to check connectivity.
- **`reset`** (or **`destroy`**) – Remove the namespace from EDA, destroy the Clab topology, and clear the stage directory.
- **`help`** – List main commands.
- **`debug`** – Show debug subcommands (or run **`debug <subcommand>`**).
- **`exit`** – Quit.

**Debug commands** (run as **`debug <subcommand>`**):

- **`debug print-settings`** – Print config and credentials in one view (passwords and secrets censored).
- **`debug print-state`** – Show which stage the process is in and what you can do next.
- **`debug set-log-level`** – Set log level at runtime (DEBUG, INFO, WARNING, ERROR).
- **`debug run-parse`** – Run Step 1 only: parse Excel topology to YAML.
- **`debug run-preprocess`** – Run Step 2 only: deploy Clab, update toponode addresses, clone index pool data.
- **`debug run-upload`** – Run Step 3 only: push resources to EDA.

Type **`debug`** with no arguments to see the debug menu. Use **`debug set-log-level`** to change log verbosity (DEBUG, INFO, WARNING, ERROR) at runtime without restarting.

## Usage

After you have the XLS templates, `config.yaml`, and `.env` in place and have edited fields as desired (keeping XLS table shape intact), run **`start`** to execute the full pipeline (parse → preprocess → upload). When prompted, provide the path to your XLS file. Use **`debug print-state`** to see current status, **`debug set-log-level`** to adjust logging, **`help`** for main commands, and **`debug`** for all debug commands.
