Metadata-Version: 2.4
Name: autotouch-cli
Version: 0.2.1
Summary: Autotouch Smart Table CLI
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.31.0
Requires-Dist: python-dotenv>=1.0.0

# Smart Table Documentation

This README is the front door to the `docs/` folder. Treat it as both the table of contents and the checklist that tells you which neighbouring docs must be updated when you change the product.

## How to use this README

- Start with **Where to look depending on the work** to route yourself to the right doc set before opening the repo.
- Use **Directory Map** when you need the canonical path for a document or when you are adding a new area of knowledge.
- Keep the sections below in sync: if you add a doc, rename a directory, or ship a new workflow, update this file in the same PR so the next contributor lands in the right place.

## Where to look depending on the work

- **Backend services or new integrations** - Begin with [development/backend/README.md](development/backend/README.md) for the target package layout, then confirm router/auth conventions + the canonical structure diagram in [platform/endpoint-patterns.md](platform/endpoint-patterns.md).
  - **MongoDB query patterns (paved road)**: see [development/backend/mongo-paved-road.md](development/backend/mongo-paved-road.md) for preferred lookup/prefetch patterns. For production access and safe querying, see [operations/mongodb-access.md](operations/mongodb-access.md).
  - **Sequence reply handling (Gmail/Outlook)**: use [systems/sequences.md](systems/sequences.md) for webhook/polling flow, `reply_tracking` correlation, `stopOnReply`, and enrollment exit behavior.
- **Frontend components, hooks, or shared UI patterns** - Start with [development/frontend/README.md](development/frontend/README.md), then review [development/frontend/component-patterns.md](development/frontend/component-patterns.md) and [development/frontend/type-system.md](development/frontend/type-system.md) before moving domain types.
  - Enrichment dialogs now share a shell + column-picker pattern (documented in component-patterns). When updating Email/Phone/Lead/LLM modals, use `EnrichmentDialogShell` and `ColumnPicker` from shared UI to keep sizing, headers, and data-aware dropdowns consistent.
  - Sequences now use the same workspace source selector pattern (research table chip) as Leads/Tasks; keep that wiring aligned with component-patterns.
- **Assist sidecar (Tasks) or JSON rendering** - Consult [development/frontend/sidecar-architecture.md](development/frontend/sidecar-architecture.md) for layout/interaction rules (including Conditional Smart Pins) and the current persistence model (`/api/sidecar/preferences` + local cache fallback), then [development/frontend/json-field-renderers.md](development/frontend/json-field-renderers.md) for typed renderers.
- **Notes / call disposition behavior** - See [platform/tasks-workspace.md](platform/tasks-workspace.md) (Tasks session orchestration, Sidecar context + lead notes), [features/immediate-followup.md](features/immediate-followup.md) (Call -> immediate manual follow-up pivot), and [data/tasks.md](data/tasks.md) (task completion + dialer dispositions).
- **Company insights / signals (dismiss/check feed)** - See [platform/insights.md](platform/insights.md) for the `insight_events` schema, endpoints, and call-sourced extraction flow. This is the foundation for future automated company signals (hiring, website, social).
- **AI drafts + step memory** - Start with [platform/drafts.md](platform/drafts.md) for the draft generator + payload mapping, then use [platform/ai-draft-send-tracking.md](platform/ai-draft-send-tracking.md) for how edits/sends refresh step memory (`step_training_profiles`) and how `_memory` is injected on subsequent drafts.
- **Usage/computing review** - See [platform/usage-and-compute-review.md](platform/usage-and-compute-review.md) for the current playbook to review billing usage, preview tracing, and compute telemetry. Supporting docs: [platform/credits.md](platform/credits.md), [development/backend/enrichment-service-usage.md](development/backend/enrichment-service-usage.md), [enrichment/preview/preview-and-improve.md](enrichment/preview/preview-and-improve.md), and [operations/azure-migration.md](operations/azure-migration.md) (for production log evidence).
- **Research table -> Tasks/Sequences flows** - Pair [platform/tasks-workspace.md](platform/tasks-workspace.md) with [systems/sequences.md](systems/sequences.md) and [features/immediate-followup.md](features/immediate-followup.md) plus the `research-table/` docs listed below to keep research_context and post-call follow-up behavior aligned.
- **Enrichment/LLM verification pipelines** - Review the `enrichment/` guides (preview, agent skills/overview) and [enrichment/hallucination-prevention.md](enrichment/hallucination-prevention.md) before modifying agent orchestration.
- **Operations, deployments, and worker health** - Start with [operations/README.md](operations/README.md) and [operations/deployment-quickstart.md](operations/deployment-quickstart.md), then pair with `workers/` queue docs (moving into `operations/`) to troubleshoot clusters and Celery topology. Worker tasks are now domain-split under `apps/workers/tasks/` (communications, enrichment `{llm|research|formula}`, leads `{finders|ingest|export}`, data_io, email schedulers, orchestration, ops); keep `apps/workers/tasks/README.md` updated when moving or rerouting tasks. Backend helpers: use `apps/workers/utils/` (shared/API-safe) or `apps/workers/utils/common/` (worker-only). Frontend/system helpers stay under their system (e.g., `apps/web/src/systems/*/utils/`).
- **Support / internal-admin workflows (impersonation)** - See [platform/authentication.md](platform/authentication.md#internal-admin-impersonation) and [operations/demo-accounts.md](operations/demo-accounts.md). Note: impersonation is org-context, but some user-scoped features (notably the dialer) require a real user in the target org.
- **Twilio media streams + transcription** - Start with [integrations/twilio/media-streams-transcription.md](integrations/twilio/media-streams-transcription.md), then cross-check [integrations/dialer-integration.md](integrations/dialer-integration.md) and [integrations/twilio/overview.md](integrations/twilio/overview.md). Includes transcription gating + wrap‑up timing anchor rules.
- **Lead contact channel status (bad phone / bounced email)** - Use [integrations/dialer-integration.md](integrations/dialer-integration.md), [data/leads.md](data/leads.md), and [data/email.md](data/email.md). Canonical backend writes live in `apps/api/services/leads/contact_status.py`.
  - Monthly ContactOut provider QA workflow (hard Twilio vs soft-signal review) is in [operations/contactout-monthly-phone-quality-review.md](operations/contactout-monthly-phone-quality-review.md).
- **Import flows (CSV / Webhook / CRM / Leads)** - Start with [platform/import-patterns.md](platform/import-patterns.md) for the shared import shell, status contract, and anti-flood guardrails. Then use [systems/leads-imports.md](systems/leads-imports.md), [integrations/crm-imports.md](integrations/crm-imports.md), and [platform/realtime-events.md](platform/realtime-events.md) for domain behavior and event contracts. Keep shared status/polling infrastructure in `apps/api/services/shared/imports/` and `apps/web/src/shared/imports/`; keep row/domain ingestion logic under the domain service/worker packages.
- **Domain modeling questions (leads, companies, activities)** - Check `data/` research notes (including `research-table/guides/add-to-leads.md`) before adjusting schemas or contracts.
- **Agent/API integrations (developer keys, scopes, table automation)** - Start with [platform/authentication.md](platform/authentication.md#developer-api-keys) for key/scoping auth rules, then [research-table/reference/tables-api.md](research-table/reference/tables-api.md#developer-api-workflow-endpoints) for the executable endpoint workflow and [platform/credits.md](platform/credits.md) for billable/non-billable behavior.
  - CLI distribution + local auth bootstrap (`autotouch auth set-key`, `pipx install autotouch-cli`) are documented in [research-table/reference/tables-api.md](research-table/reference/tables-api.md#public-cli-quickstart-autotouch).

## Directory Map (Where to Start)

- [platform/](platform/) - Core platform behavior (auth, tenancy, API conventions, Smart Table structure, end-to-end prospecting lifecycle).
  - [platform/authentication.md](platform/authentication.md) - Auth system, developer API keys/scopes, tenancy rules, and internal-admin impersonation.
  - [platform/credits.md](platform/credits.md) - Credit policy (what charges when), ledger invariants, and UI semantics on insufficient balance.
  - [platform/email/](platform/email/) - Email management (blacklist, policies). See [platform/email/blacklist.md](platform/email/blacklist.md).
  - [platform/endpoint-patterns.md](platform/endpoint-patterns.md) - Canonical backend routing/auth/filtering patterns, including Sequences orchestrator/scheduler/webhooks.
    - [platform/permissions.md](platform/permissions.md) - Permission matrix, enforcement helpers, rollout checklist.
    - [platform/realtime-events.md](platform/realtime-events.md) - Socket event contracts, including bulk operation progress payloads.
  - [platform/import-patterns.md](platform/import-patterns.md) - Canonical import architecture: shared shell + status contract, domain-specific processors, and anti-flood defaults.
  - [platform/tasks-workspace.md](platform/tasks-workspace.md) - Tasks workspace behavior (research-column formatting, scope defaults, research-table restrictions when creating tasks).
    - [systems/sequences.md](systems/sequences.md) - Legacy "Add to campaign" sync: provider detection, merge-variable mapping, Instantly/Smartlead webhook flows.
- [platform/ai_company_research.md](platform/ai_company_research.md) - AI Company Research pipeline: forked list-builder flow, condition columns, and neural-search batching rules.
- [platform/insights.md](platform/insights.md) - Company insights feed (`insight_events`): dismiss/check UX, endpoints, and call-sourced extraction flow (foundation for future company signals).
- [platform/drafts.md](platform/drafts.md) - Draft generation service used by sequences/tasks (LLM meta prompts + task drafts).
- [platform/ai-draft-send-tracking.md](platform/ai-draft-send-tracking.md) - How AI drafts are tracked (generate vs send/complete) and how edits refresh step memory used by `_memory`.
- [systems/](systems/) - System-specific architecture docs and feature wiring.
  - [systems/README.md](systems/README.md) - Entry point for system docs.
  - [systems/sequences.md](systems/sequences.md) - Sequences tool architecture, endpoints, scheduler/webhook scaffold, and the Instantly/Smartlead provider bridge.
    - Enrollment now persists `research_context` plus `research_snapshot.row_id` per lead; integration mode pushes normalized variables into provider campaigns.
  - [systems/leads-crm.md](systems/leads-crm.md) - Leads CRM workspace behavior, research context, and UI decisions.
  - [systems/leads-imports.md](systems/leads-imports.md) - CSV lead imports, mapping, dedupe, and company linkage.
  - [systems/leads-query-endpoint.md](systems/leads-query-endpoint.md) - Leads query endpoint contract and filtering payloads.
- [development/](development/) - Engineering practices and coding patterns.
  - [development/backend/](development/backend/) - Service architecture and migration guidance; start with [development/backend/README.md](development/backend/README.md) when adding/moving services.
    - [development/backend/chat-service.md](development/backend/chat-service.md) - Provider-agnostic streaming Chat service (SSE message frames, adapters, env vars, persistence roadmap).
    - [development/backend/mongo-paved-road.md](development/backend/mongo-paved-road.md) - Preferred MongoDB lookup/prefetch patterns (avoid N+1, keep projections tight, index-friendly query shapes).
    - [development/backend/enrichment-service-usage.md](development/backend/enrichment-service-usage.md) - Unified preview streaming schema and entry points.
    - [development/backend/helpers.md](development/backend/helpers.md) - Shared backend helpers (research snapshots, context providers, lead/company services) plus the unified enrichment schema used by drawers and cells.
      - [development/backend/mcp-tools-integration.md](development/backend/mcp-tools-integration.md) - MCP tool integration + registry scaffolding for chat tool frames.
      - [development/backend/review-checklist.md](development/backend/review-checklist.md) - Backend checklist covering endpoints, streaming, limits, observability, security.
  - [development/frontend/](development/frontend/) - System-first architecture references; start with [development/frontend/README.md](development/frontend/README.md).
    - [development/frontend/README.md](development/frontend/README.md) - Frontend placement checklist and doc map.
  - [development/frontend/component-patterns.md](development/frontend/component-patterns.md) - Table/toolbar architecture and shared checkbox behavior across research/leads/tasks.
  - [development/frontend/command-palette.md](development/frontend/command-palette.md) - Global Cmd+K command palette roadmap and command-bar integration plan.
  - [development/frontend/type-system.md](development/frontend/type-system.md) - Domain-driven type layout, audit checklist, maintenance guidance.
  - [development/frontend/table-layout-persistence.md](development/frontend/table-layout-persistence.md) - How table layouts/sort/filters persist to localStorage and how to version/reset defaults.
    - [development/frontend/sidecar-architecture.md](development/frontend/sidecar-architecture.md) - Assist sidecar geometry, Lead-first tabs, close/toggle behavior, AI chat composition, and Conditional Smart Pins (MVP rules).
    - [development/frontend/sidecar-meta-prompt.md](development/frontend/sidecar-meta-prompt.md) - Meta prompt for continuing the Assist Sidecar + AI Chat implementation.
    - [development/frontend/json-field-renderers.md](development/frontend/json-field-renderers.md) - Typed renderers, array slice chips, object micro-grids, raw JSON overlay, copy actions.
    - [development/frontend/review-checklist.md](development/frontend/review-checklist.md) - Frontend checklist for sidecar/chat (geometry, perf, a11y, telemetry).
    - [development/frontend/ai-elements.md](development/frontend/ai-elements.md) - Composition docs for AI chat UI primitives.
  - [development/patterns.md](development/patterns.md) - End-to-end workflow tying backend and frontend practices together.
- [research-table/](research-table/) - Everything specific to the research table experience (concepts, guides, reference).
  - [concepts/pinned-columns.md](research-table/concepts/pinned-columns.md) - Sticky/pinned column behavior, hover/shadow rules, and localStorage persistence per table.
  - [guides/object-drawer.md](research-table/guides/object-drawer.md) - Row detail drawer interaction (hover swap, keyboard open, grouped vertical layout).
  - [guides/add-to-leads.md](research-table/guides/add-to-leads.md) - Add-to-Leads flow, field mapping, and worker orchestration.
  - [concepts/formatter-columns.md](research-table/concepts/formatter-columns.md) - JavaScript formatter columns (AI + manual formulas, dependency tracking, hybrid execution engine).
  - [concepts/projection-columns.md](research-table/concepts/projection-columns.md) - JSON splits (virtual by default; optional materialization for fast filter/sort) with API contract, filtering, and export behavior.
  - Reports (implementation notes and migrations):
    - [reports/materialized-projection-columns.md](research-table/reports/materialized-projection-columns.md) - Materialized projections rollout notes (jobs, queues, routing, observability).
  - [reference/research-context-coverage.md](research-table/reference/research-context-coverage.md) - `/research-context` coverage calculation, nested projection handling, limitations.
  - [reference/filtering.md](research-table/reference/filtering.md) - End-to-end filtering semantics, operator matrix, Mongo vs Python routing, and column-run scoping (`filtered` vs `subset`) with the centralized run-scope resolver.
  - [reference/formatter-editing.md](research-table/reference/formatter-editing.md) - Canonical formatter storage (`config.formula`), editing surfaces, lookup normalization, preview/run behavior, and troubleshooting.
  - [reference/tables-api.md](research-table/reference/tables-api.md) - Table + column endpoints (column keys are unique per table; the backend owns suffixing and returns the final key; 409 is returned on conflicts).
  - Socket ingress note: see [platform/realtime-events.md](platform/realtime-events.md#common-issues) - `/socket.io/*` must route to the API service in split ingress setups.
  - Keep enrichment columns aligned with the unified schema in [development/backend/helpers.md](development/backend/helpers.md) so drawers render Steps/details consistently; see [enrichment/hallucination-prevention.md](enrichment/hallucination-prevention.md) for verification rules.
  - Research Agent & Generator references live under `enrichment/`:
    - [enrichment/concepts/prompt-generator-concepts.md](enrichment/concepts/prompt-generator-concepts.md)
    - [enrichment/guides/prompt-generator-usage.md](enrichment/guides/prompt-generator-usage.md)
    - [enrichment/reference/react-kernel.md](enrichment/reference/react-kernel.md)
    - [enrichment/reference/assembler.md](enrichment/reference/assembler.md)
    - [enrichment/reference/context-pipeline.md](enrichment/reference/context-pipeline.md)
    - [enrichment/reference/placeholder-policy.md](enrichment/reference/placeholder-policy.md)
    - [enrichment/reference/verification-pipeline.md](enrichment/reference/verification-pipeline.md)
- [integrations/](integrations/) - External system integrations (Aircall, RingCentral, auto-dialer).
  - [integrations/dialer-integration.md](integrations/dialer-integration.md) - Auto-dialer backend integration: session/queue model, provider registry, endpoints.
  - [integrations/twilio/media-streams-transcription.md](integrations/twilio/media-streams-transcription.md) - Twilio media streams + AssemblyAI transcription pipeline, storage, and training format.
  - [integrations/twilio/](integrations/twilio/) - Managed Twilio dialer integration and local presence notes.
  - [integrations/crm-imports.md](integrations/crm-imports.md) - CRM imports (Attio/HubSpot) status, config, and next steps for Research-table -> CRM workflows.
  - [integrations/crm-roadmap.md](integrations/crm-roadmap.md) - CRM long-term vision and roadmap (imports now, exports/backfill, sync, activities later).
- [operations/](operations/) - Runbooks covering deployment, monitoring, and worker operations.
  - [operations/deployment-quickstart.md](operations/deployment-quickstart.md) - Build/deploy, auth keys, routing, autoscaling, and post-deploy checks.
  - [operations/contactout-monthly-phone-quality-review.md](operations/contactout-monthly-phone-quality-review.md) - Monthly ContactOut phone quality QA workflow (exports, hard evidence vendor packet, soft-signal analysis).
  - [operations/azure-migration.md](operations/azure-migration.md) - AKS/ACR/Ingress/Redis migration status + runbook.
    - Backend log access (AKS + Log Analytics) is documented under `operations/azure-migration.md#monitoring-aks--logs`.
    - Incident audit (time-window logs, Mongo/Atlas correlation, checklist): [operations/incident-audit.md](operations/incident-audit.md).
  - [operations/hardware.md](operations/hardware.md) - Current AKS hardware snapshot (nodes, allocatable, scheduling headroom).
  - [operations/mongodb-migration-2025-11.md](operations/mongodb-migration-2025-11.md) - Atlas move (GCP -> Azure), current DB layout, and connection guidance.
  - [operations/mongodb-access.md](operations/mongodb-access.md) - How to connect/query Mongo in production safely (uses `MONGODB_PROD_CONNECTION_STRING`).
  - [operations/mongodb-ssl-handshake-troubleshooting.md](operations/mongodb-ssl-handshake-troubleshooting.md) - SSL/TLS handshake failures troubleshooting, connection pool tuning, and retry logic fixes.
  - Slack monitoring: [operations/slack-error-logging.md](operations/slack-error-logging.md) (setup + config, automatic coverage, and credit alerts).
- [workers/](workers/) - Background job notes and queue-specific documentation (moving into `operations/`).
  - [workers/celery.md](workers/celery.md) - Celery topology (communications/enrichment/leads/data_io/ops queues), deployment commands, routing patterns aligned with the domain-split worker tree.
  - [workers/bulk-jobs.md](workers/bulk-jobs.md) - Bulk job orchestration (providers, status tracking, progress updates, adding a provider).
  - Worker code layout: `apps/workers/tasks/README.md` (domain map + routing notes).
- [data/](data/) - Domain write-ups about leads, companies, activities, and related data-model research.
  - [research-table/guides/add-to-leads.md](research-table/guides/add-to-leads.md) - How research tables pipe leads into the CRM.
- [enrichment/](enrichment/) - Contact enrichment overviews and guides.
  - [enrichment/README.md](enrichment/README.md) - Entry point for enrichment docs.
  - [enrichment/pipeline-conventions.md](enrichment/pipeline-conventions.md) - Shared real-time events, credits, and schema conventions for pipelines.
  - [enrichment/preview/preview-and-improve.md](enrichment/preview/preview-and-improve.md) - LLM preview + coach architecture, agent skills for research.
  - [enrichment/prompt-templates.md](enrichment/prompt-templates.md) - Prompt library templates (simple vs raw), mode/scope filtering, save/load behavior, and API shape.
  - [enrichment/skills/README.md](enrichment/skills/README.md) - Research agent skills spec (YAML), discovery model, selector/assembler overview, authoring tips.
    - [enrichment/SYSTEM_ARCHITECTURE.md](enrichment/SYSTEM_ARCHITECTURE.md) - Research agent architecture (ReAct loop, skills, MCP gateway/tools) and common pitfalls.
    - [enrichment/research-agent-overview.md](enrichment/research-agent-overview.md) - Research agent one-pager (skills gating, MCP tools, preview vs batch, validation, config).
    - [enrichment/hallucination-prevention.md](enrichment/hallucination-prevention.md) - LLM hallucination detection/prevention pipeline.
    - [enrichment/AUDIT_HALLUCINATION_PREVENTION.md](enrichment/AUDIT_HALLUCINATION_PREVENTION.md) - Meta-prompt for auditing verification strictness.
    - [enrichment/schema-generator.md](enrichment/schema-generator.md) - How `/api/llm/generate-schema` works, reasoning-field rules, and when auto-generated schemas are used.
    - [enrichment/guides/provider-overrides.md](enrichment/guides/provider-overrides.md) - How to request specific providers/models when calling the MCP gateway.
    - [enrichment/phone_finder/overview.md](enrichment/phone_finder/overview.md) - Phone discovery pipeline and Leads mapping.
    - [enrichment/email_finder/overview.md](enrichment/email_finder/overview.md) - Email discovery + verification flow and Leads mapping.
    - [enrichment/email_validation/overview.md](enrichment/email_validation/overview.md) - Validation-only pipeline with positional updates.
    - [enrichment/lead_finder/overview.md](enrichment/lead_finder/overview.md) - Contact discovery at target companies (provider limits, retry strategy).
      - [enrichment/lead_finder/research-table.md](enrichment/lead_finder/research-table.md) - Lead Finder as a research-table enrichment column (submission/parallel env controls, rate-limit handling).
      - [enrichment/lead_finder/leads-crm.md](enrichment/lead_finder/leads-crm.md) - Bulk enrichment actions in the Leads CRM.
- [infra/](infra/) - Infrastructure provisioning notes and migration anchors.
  - [infra/README.md](infra/README.md) - Entry point for infra docs.
  - [infra/azure/README.md](infra/azure/README.md) - Azure provisioning inventory and commands.
- [meta/](meta/) - Temporary prompts, audits, and planning notes (not canonical product docs; intentionally not indexed here).

When you introduce a new document, decide where it belongs in this map and add (or update) the entry in the same pull request. If a directory description stops matching reality, fix it here while you are already in context.

## Keeping Docs Manageable

- Prefer focused documents over encyclopedias; split long write-ups into concept, guide, and reference files when possible.
- Cross-link related docs instead of duplicating large sections of content.
- Trim outdated context when updating a file; point to source code or dashboards for canonical details.

Whenever you touch the docs, glance at this README to confirm the structure still makes sense. Update it in the same pull request if the hierarchy shifts or if you create a new area so other contributors can navigate quickly.

## Keeping this README Accurate

- Update the **Directory Map** entry (and add new bullets) whenever you add, remove, or rename docs/directories.
- Extend **Where to look depending on the work** when a new workflow spans multiple areas so future contributors know which sibling docs to audit.
- Verify every link you touch in this file; keep them relative and clickable so the README remains the fastest navigation surface.
- Call out "docs README updated" in your PR description when you touch this file; reviewers should confirm the map still mirrors the repo.
