Metadata-Version: 2.4
Name: alvitr-sentinel
Version: 1.0.1
Summary: Automated Scientific Literature Monitoring System
Author: AlvitrSentinel Contributors
License: MIT
Keywords: literature,pipeline,rss,ai,research,academic,monitoring,bioinformatics
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Text Processing :: Markup :: HTML
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: feedparser>=6.0
Requires-Dist: requests>=2.28
Requires-Dist: openai>=1.0
Requires-Dist: jinja2>=3.1
Requires-Dist: pyyaml>=6.0
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: lxml>=4.9
Requires-Dist: tqdm>=4.65
Dynamic: license-file

# AlvitrSentinel

**Automated Scientific Literature Monitoring System**

AlvitrSentinel is a command-line tool that automates the daily monitoring of scientific literature. It fetches articles from RSS/Atom feeds of major journals, uses AI to generate summaries, classify topics, and score relevance to your research interests, then produces interactive HTML reports for convenient review and manual archiving to reference managers such as Zotero.

## Skills
I also support AI skills in AlvitrSentinelSkills.

## Features

- **RSS/Atom Feed Fetching** — Supports ETag / If-Modified-Since conditional requests for efficient caching, automatic retries with exponential back-off, and configurable timeouts.
- **Three-Tier Abstract Acquisition** — (1) RSS-embedded description, (2) web-page scraping of the abstract section, (3) title-only fallback with appropriate labeling.
- **AI-Powered Processing** — Batch LLM calls via any OpenAI-compatible API to generate Chinese-language summaries, assign topic tags, and produce a 1-5 relevance score with rationale.
- **Dual HTML Reports** — A daily briefing stratified by relevance tier, and a cumulative single-page dashboard with full-text search, date-range filtering, topic filtering, and paginated results.
- **Optional Embedding Vectors** — Compute dense vectors for semantic search and article clustering using a separate embedding model.
- **Pure File-Based Storage** — JSON files partitioned by date; no database required.
- **Flexible Configuration** — YAML config covering AI models, research interests, topic taxonomy, feed list, and all pipeline parameters.

## Installation

```bash
pip install alvitr-sentinel
```

Or install from source:

```bash
git clone https://github.com/Secretloong/AlvitrSentinel.git
cd alvitr-sentinel
pip install -e .
```

## Quick Start

```bash
# 1. Initialize project (creates directories, config, and feed list)
alvitr-sentinel --init

# 2. Edit configuration
#    - Set your LLM API key
#    - Describe your research interests
#    - Customize the feed list
vim config.yaml
vim journal-feeds.txt

# 3. Run the full pipeline
alvitr-sentinel
```

## Command-Line Usage

| Command | Description |
|---------|-------------|
| `alvitr-sentinel` | Full pipeline run |
| `alvitr-sentinel --init` | Initialize project directory |
| `alvitr-sentinel --fetch-only` | Fetch and parse only, skip AI (useful for testing) |
| `alvitr-sentinel --no-ai` | Skip AI processing, still store and generate reports |
| `alvitr-sentinel --report-only` | Regenerate reports from existing data |
| `alvitr-sentinel --date 2026-03-20` | Process a specific date |
| `alvitr-sentinel --days 60` | Override dashboard display window |
| `alvitr-sentinel -v` | Enable verbose logging |
| `alvitr-sentinel --version` | Show version |

## Configuration

The `config.yaml` file controls all aspects of the pipeline:

| Section | Key Fields | Description |
|---------|-----------|-------------|
| `ai.llm` | `base_url`, `api_key`, `model` | LLM inference model for summarization and scoring |
| `ai.embedding` | `enabled`, `base_url`, `model` | Optional embedding model for semantic search |
| `research_interests` | (free text) | Natural-language description of your research focus; directly affects relevance scoring |
| `topic_taxonomy` | (list) | Topic labels the AI assigns to each article |
| `filter` | `min_relevance`, `highlight_threshold` | Controls report filtering and highlighting |
| `fetcher` | `timeout`, `retries`, `user_agent` | Feed fetching behavior |
| `parser` | `min_abstract_length`, `scrape_abstract` | Abstract extraction settings |

## Pipeline Architecture

```
journal-feeds.txt
       |
       v
  +---------+     +---------+     +-----------+     +---------+     +--------+
  | Fetcher | --> | Parser  | --> | AIEngine  | --> | Storage | --> | Report |
  +---------+     +---------+     +-----------+     +---------+     +--------+
  RSS/Atom        Metadata         Summarize,        JSON by         HTML
  fetch with      extraction,      classify,         date,           daily +
  ETag cache      3-tier           score             dedup index     dashboard
                  abstract
```

## Output

- **Daily Briefing**: `data/reports/daily/YYYY-MM-DD.html` — Articles grouped by relevance tier (high / medium / low) with AI summaries, topic tags, and direct links to original papers.
- **Cumulative Dashboard**: `data/reports/dashboard.html` — A self-contained single-page application with search, filtering, sorting, and pagination across all collected articles.

## Supported AI Services

Any OpenAI-compatible API endpoint works, including:

- Alibaba Cloud DashScope (Qwen series)
- SiliconCloud / SiliconFlow
- OpenAI
- Azure OpenAI
- Local LLM servers (vLLM, Ollama, etc.)

## Module Overview

| Module | Description |
|--------|-------------|
| `fetcher.py` | RSS/Atom feed fetcher with conditional request caching and retry logic |
| `parser.py` | Article metadata extraction with three-tier abstract acquisition |
| `ai_engine.py` | AI summarization, classification, and relevance scoring engine |
| `storage.py` | JSON file storage layer with deduplication index |
| `report_generator.py` | Jinja2-based HTML report generator (daily briefing + dashboard) |
| `pipeline.py` | CLI entry point and pipeline orchestrator |

## Requirements

- Python >= 3.9
- feedparser, requests, openai, jinja2, pyyaml, beautifulsoup4, lxml, tqdm

## License

MIT License
