feat(media-routing): implement 007 media article routing, runtime architecture diagram and update graphify knowledge graph

.specify/memory/constitution.md

@@ -1,50 +1,61 @@
-# [PROJECT_NAME] Constitution
-<!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
+<!--
+Sync Impact Report
+Version change: Initial Template -> 1.0.0
+Modified principles: Initialized all Core Principles from template placeholders
+Added sections:
+  - Core Principles:
+    - I. Modularity & CLI-First Interoperability
+    - II. Determinism, Atomic Operations & Data Integrity
+    - III. Multi-Engine Consensus & Fault-Tolerant Fallback
+    - IV. Test-First & Empirical Validation
+    - V. Observability, Structured Logging & Traceability
+  - Technical, Security & Environmental Constraints
+  - Development Workflow & Quality Gates
+  - Governance & Amendment Protocol
+Removed sections: None
+Follow-up TODOs: None
+-->
+
+# TextNLPClassifierApp Constitution
 
 ## Core Principles
 
-### [PRINCIPLE_1_NAME]
-<!-- Example: I. Library-First -->
-[PRINCIPLE_1_DESCRIPTION]
-<!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
+### I. Modularity & CLI-First Interoperability
+Every capability (crawling, multi-engine extraction, consensus selection, markdown conversion, NLP/LLM classification, and consolidation runtime) MUST be implemented as a modular, decoupled component under `src/` or `scripts/`. Every core component MUST expose a standard CLI interface supporting both human-readable logging and structured JSON input/output over standard streams, with explicit exit codes for automated pipeline orchestration.
 
-### [PRINCIPLE_2_NAME]
-<!-- Example: II. CLI Interface -->
-[PRINCIPLE_2_DESCRIPTION]
-<!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
+### II. Determinism, Atomic Operations & Data Integrity
+Text processing, shingle extraction, $F_1$ consensus calculation, metadata matrix resolution, and formatting transformations MUST be strictly deterministic, reproducible, and idempotent. All filesystem writes MUST execute via atomic transactional write patterns (temporary file staging followed by atomic rename) to eliminate corrupted state. Source data and multi-extractor payloads MUST be preserved non-destructively.
 
-### [PRINCIPLE_3_NAME]
-<!-- Example: III. Test-First (NON-NEGOTIABLE) -->
-[PRINCIPLE_3_DESCRIPTION]
-<!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
+### III. Multi-Engine Consensus & Fault-Tolerant Fallback
+Data extraction and parsing MUST NOT rely on single points of failure. The architecture enforces multi-engine extraction (Trafilatura, Newspaper4k, Readability) coupled with automated consensus scoring ($F_1$ 5-token shingles) and strict fallback heuristics. Edge cases (paywalls, empty bodies, anti-bot challenges) MUST be handled gracefully with explicit diagnostics.
 
-### [PRINCIPLE_4_NAME]
-<!-- Example: IV. Integration Testing -->
-[PRINCIPLE_4_DESCRIPTION]
-<!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
+### IV. Test-First & Empirical Validation (TDD & Regressions)
+Test-Driven Development (TDD) and empirical test suites are mandatory. Unit tests, integration contracts, and regression suites MUST cover parser algorithms, CLI flags, exit codes, and error conditions before deployment. A 100% passing test baseline MUST be maintained (`pytest`), accompanied by static linting (`ruff`) and static type validation (`mypy`/`pyright`).
 
-### [PRINCIPLE_5_NAME]
-<!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
-[PRINCIPLE_5_DESCRIPTION]
-<!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
+### V. Observability, Structured Logging & Traceability
+All pipeline phases (crawling, article selection, consolidation, LLM/NLP classification) MUST emit structured, contextual telemetry. Operations MUST log execution metrics, engine scores, token consumption, and decision paths. Tracing integrations (e.g., Langfuse) MUST provide full visibility into prompt performance, latency, cost, and classification rationale.
 
-## [SECTION_2_NAME]
-<!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
+## Technical, Security & Environmental Constraints
 
-[SECTION_2_CONTENT]
-<!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
+- **Language & Runtime**: Python `>=3.10` with strict type annotations across all modules.
+- **Code Quality**: Linting and formatting governed by `ruff` (100-character line limit) and type safety verified via `mypy`.
+- **Security & Secrets**: Zero hardcoded credentials or API keys; configuration MUST be loaded from environment variables (`.env`) with schemas documented in `.env.example`.
+- **Stealth & Web Automation**: Browser automation (Foxcape/Camoufox) MUST enforce appropriate delays, backoff, and fingerprint management without exceeding target service rate limits.
 
-## [SECTION_3_NAME]
-<!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
+## Development Workflow & Quality Gates
 
-[SECTION_3_CONTENT]
-<!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
+- **Specification First**: Features and structural changes MUST be documented in specifications (`specs/` or `docs/`) with clear functional requirements and architecture decision records (ADRs).
+- **Mandatory Quality Gates**: Every code contribution MUST pass:
+  1. `ruff check` and `ruff format --check` (clean formatting and linting).
+  2. `mypy` / `pyright` (no type check violations).
+  3. `pytest` (full test suite execution with 100% passing rate).
+- **Commit Standards**: Atomic commits using Conventional Commits convention (`feat:`, `fix:`, `docs:`, `test:`, `refactor:`, `chore:`).
 
 ## Governance
-<!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
 
-[GOVERNANCE_RULES]
-<!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
+This Constitution represents the supreme architectural and development policy for `TextNLPClassifierApp`. All development, automated subagents, and pull requests MUST comply with the principles and quality gates established herein. Amendments require explicit documentation of rationale, team review, and formal version increments:
+- **MAJOR**: Incompatible principle removals or foundational architectural shifts.
+- **MINOR**: Addition of new principles, governance rules, or expanded standards.
+- **PATCH**: Wording improvements, clarifications, and non-semantic corrections.
 
-**Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
-<!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
+**Version**: 1.0.0 | **Ratified**: 2026-08-24 | **Last Amended**: 2026-08-24