# Feature Specification: Multilingual NLP Entity Inherence Classifier (POC)

**Feature Branch**: `001-multilingual-entity-classifier`

**Created**: 2026-08-19

**Status**: Draft / Documented (POC Scope)

**Input**: User description: "Ferramenta de NLP em vários idiomas (português, inglês, espanhol, alemão, italiano, francês). Recebe um conteúdo em Markdown e um Entity Context Profile (ECP Snapshot em JSON), analisando se aquele conteúdo é inerente àquela entidade."

---

## Clarifications

### Session 2026-08-19

- Q: O que é o ECP e qual o formato de entrada esperado? → A: **ECP significa Entity Context Profile**. A entrada para o classificador é um **ECP Snapshot materializado em formato JSON** (autocontido, com entidade alvo, aliases, anchors e entidades relacionadas do grafo). O conteúdo a ser avaliado é fornecido em formato **Markdown**.
- Q: Qual a interface e escopo da POC? → A: A ferramenta de classificação roda exclusivamente como um **script Python CLI** (`python classify.py --ecp <snapshot.json> --content <doc.md> --output <result.json>`). Ficam fora de escopo para a POC: REST API, pacote publicável, filas/workers e integrações em runtime com banco de grafos (Neo4j) ou ContentMachineApp.
- Q: Como o pipeline híbrido deve se comportar na POC? → A: O **Tier 1 (regras determinísticas)** é o núcleo obrigatório da POC. O **Tier 2 (embeddings multilíngues locais)** é opcional/configurável. O **Tier 3 (LLM fallback)** é opcional/configurável e **desativado por padrão** (casos óbvios não chamam LLM e nenhuma API key externa é necessária para executar a POC).
- Q: Quais as regras mínimas de decisão e o formato de saída? → A: Decisão categorizada em 4 níveis (`DIRECT_INHERENT`, `CONTEXTUAL_INHERENT`, `TANGENTIAL`, `NOT_RELATED`) com `is_inherent` como campo derivado booleano (`true` para `DIRECT_INHERENT` e `CONTEXTUAL_INHERENT`; `false` para `TANGENTIAL` e `NOT_RELATED`), acompanhado de `confidence`, `matched_anchors`, `negative_matches`, `graph_matches`, `evidence`, `rationale` e `warnings`.
- Q: Como deve ser o benchmark de validação da POC? → A: Um conjunto de teste controlado com no mínimo **24 casos (6 idiomas × 4 tipos de decisão)**, medindo acurácia ≥ 90% especificamente sobre essa suíte de benchmark da POC.

---

## User Scenarios & Testing *(mandatory)*

### User Story 1 - Classify Content Inherence via CLI for an ECP Snapshot (Priority: P1)

As a developer or automation script, I want to execute a Python CLI command passing an ECP Snapshot (JSON) and a document (Markdown), so that I get a structured JSON evaluation determining whether the document is inherent to the target entity.

**Why this priority**: Core value proposition and execution model of the POC.

**Independent Test**: Run `python classify.py --ecp examples/ecp_petrobras.json --content examples/content_presal_pt.md --output out/result.json` and verify the output contains `decision: "DIRECT_INHERENT"` and `is_inherent: true`.

**Acceptance Scenarios**:

1. **Given** a Portuguese Markdown article discussing offshore oil exploration and an ECP Snapshot for "Petrobras", **When** `classify.py` executes, **Then** it produces `decision: "DIRECT_INHERENT"`, `is_inherent: true`, and matched aliases in `matched_anchors`.
2. **Given** a German Markdown article about battery supply chains and an ECP Snapshot for "Volkswagen Group" with a related entity "Northvolt" in its graph snapshot, **When** evaluated, **Then** it produces `decision: "CONTEXTUAL_INHERENT"`, `is_inherent: true`, and "Northvolt" listed in `graph_matches`.
3. **Given** a Spanish Markdown text that mentions "Apple" only in a passing metaphor ("la manzana de la discordia") without tech context, **When** evaluated against an ECP Snapshot for "Apple Inc.", **Then** it produces `decision: "TANGENTIAL"`, `is_inherent: false`.
4. **Given** an Italian recipe text and an ECP Snapshot for "Ferrari N.V.", **When** evaluated, **Then** it produces `decision: "NOT_RELATED"`, `is_inherent: false`.

---

### User Story 2 - Multilingual Language Support & Edge Validation (Priority: P2)

As an evaluator, I want the CLI to handle content across 6 languages (PT, EN, ES, DE, IT, FR) and return explicit, structured error diagnostics on invalid inputs.

**Why this priority**: Guarantees baseline reliability and debuggability across all target languages.

**Independent Test**: Execute the CLI against a test suite covering the 6 languages and against corrupted/empty inputs, verifying proper decisions and error codes.

**Acceptance Scenarios**:

1. **Given** valid Markdown documents in each of the 6 languages, **When** classified against matching ECP Snapshots, **Then** the system correctly identifies `detected_language` and applies language-aware tokenization/matching.
2. **Given** a corrupted JSON file or empty Markdown file, **When** `classify.py` executes, **Then** it outputs a structured JSON error response with appropriate error codes (`invalid_ecp_json`, `empty_content`, etc.) and non-zero exit code.

---

## Requirements *(mandatory)*

### Functional Requirements

- **FR-001**: System MUST support multilingual NLP classification across at least 6 core languages: Portuguese (`pt`), English (`en`), Spanish (`es`), German (`de`), Italian (`it`), and French (`fr`).
- **FR-002**: System MUST provide a standalone Python CLI entrypoint:
  ```bash
  python classify.py --ecp <snapshot.json> --content <doc.md> --output <result.json>
  ```
  If `--output` is omitted, the result JSON MUST be emitted to standard output (`stdout`). Both `--ecp` and `--content` MUST be valid file paths.
- **FR-003**: System MUST parse and validate the materialized ECP Snapshot JSON structure:
  - **Required fields**: `target_entity_id` (string), `target_name` (string), `aliases` (array of strings), `domain` (string), `anchors` (array of strings).
  - **Optional fields with defaults**: `negative_anchors` (array of strings, default `[]`), `graph_version` (string, default `"1.0.0"`), `related_entities` (array of objects, default `[]`).
  - **Related entity structure**: `entity_id` (string), `name` (string), `relation_type` (string), `weight` (number), `aliases` (array of strings, default `[]`), `scope` (string, default `"general"`), `confidence` (number, default `1.0`).
- **FR-004**: System MUST execute a 3-tier hybrid classification pipeline:
  - **Tier 1 (Deterministic Rules - Mandatory in POC)**: Token matching for aliases, anchors, negative anchors, language detection, and graph snapshot node matches.
  - **Tier 2 (Local Multilingual Embeddings - Optional/Configurable)**: Semantic similarity scoring using local embeddings (can be toggled via config/flag).
  - **Tier 3 (LLM Fallback - Optional/Configurable & Disabled by Default)**: Invoked only for unresolved ambiguous boundary cases when explicitly enabled. Clear cases MUST NOT call LLM. No external API key is required to run the POC.
- **FR-005**: System MUST enforce the following decision logic:
  - `DIRECT_INHERENT`: Strong match of target entity aliases + compatible domain/context + no dominant negative anchors.
  - `CONTEXTUAL_INHERENT`: Match of related entity from snapshot + scope criteria satisfied + sufficient relational weight/confidence.
  - `TANGENTIAL`: Weak/isolated mention, relationship lacking surrounding context, or insufficient evidence.
  - `NOT_RELATED`: Absence of relevant signals or dominant negative anchors.
- **FR-006**: System MUST derive the boolean `is_inherent` strictly from `decision`:
  - `DIRECT_INHERENT` → `is_inherent: true`
  - `CONTEXTUAL_INHERENT` → `is_inherent: true`
  - `TANGENTIAL` → `is_inherent: false`
  - `NOT_RELATED` → `is_inherent: false`
- **FR-007**: System MUST output a success JSON containing:
  - `decision`: String (`DIRECT_INHERENT` | `CONTEXTUAL_INHERENT` | `TANGENTIAL` | `NOT_RELATED`)
  - `is_inherent`: Boolean (`true` | `false`)
  - `confidence`: Float (`0.0` to `1.0`)
  - `detected_language`: String (ISO language code)
  - `matched_anchors`: Array of strings
  - `negative_matches`: Array of strings
  - `graph_matches`: Array of objects/strings (matched related entities from snapshot)
  - `evidence`: Array of strings (excerpts extracted from the Markdown)
  - `rationale`: String (short explanation of the decision)
  - `warnings`: Array of strings
- **FR-008**: System MUST output structured error responses for failure conditions:
  - `error_code`: Enum (`invalid_ecp_json`, `invalid_markdown`, `unsupported_language`, `empty_content`, `missing_required_field`)
  - `message`: Human-readable error description
  - `details`: Object with debugging details

---

### Key Entities *(data models & domain entities)*

- **Entity Context Profile (ECP) Snapshot (JSON)**: Materialized, self-contained entity snapshot containing target entity metadata, aliases, anchors, negative anchors, and relevant connected graph entities.
- **Content Item (Markdown)**: The input markdown document payload to be analyzed.
- **Inherence Assessment Result (JSON)**: The structured result produced by the CLI containing classification verdict, derived boolean, confidence score, evidence snippets, and rationale.
- **Classification Error (JSON)**: Structured failure payload with standardized error code.

---

## Success Criteria *(mandatory)*

### Measurable Outcomes

- **SC-001**: System executes successfully via Python CLI without external network/service dependencies when running Tier 1.
- **SC-002**: Classification precision reaches at least **90% over a controlled POC benchmark suite of 24 cases** (6 languages: PT, EN, ES, DE, IT, FR × 4 decision types: `DIRECT_INHERENT`, `CONTEXTUAL_INHERENT`, `TANGENTIAL`, `NOT_RELATED`).
- **SC-003**: 100% of outputs conform strictly to the specified JSON success/error schemas.
- **SC-004**: CLI execution time for Tier 1 deterministic evaluation is under 200ms for documents under 2,000 words.

---

## Assumptions & Scope

### Assumptions
- The 6 core target languages are Portuguese (`pt`), English (`en`), Spanish (`es`), German (`de`), Italian (`it`), and French (`fr`).
- ECP Snapshots are pre-materialized JSON files (the ECP Manager / Neo4j is responsible for publishing snapshots; the classifier does not connect to Neo4j or external databases in runtime).
- Tier 1 deterministic logic is sufficient for baseline evaluation; LLM fallback is an optional enhancement for future edge-case tuning.

### Explicit Out of Scope (POC)
- REST API / FastAPI microservices.
- PyPI package creation/distribution.
- Message queues (RabbitMQ, SQS, Celery) or background workers.
- Direct runtime database or graph database connections (Neo4j).
- Integration with ContentMachineApp.
- Mandatory external LLM API keys.