feat: add deterministic content extractor selector engine with F1 consensus

2026-08-20 22:09:43 -03:00
parent 6a45368cb0
commit ff7a50e0eb
46 changed files with 18503 additions and 2813 deletions
@@ -0,0 +1,80 @@
+# JSON Schema Contract: Deterministic Article Content Selection
+
+**Branch**: `004-deterministic-content-selection` | **Date**: 2026-08-20 | **Spec**: [spec.md](../spec.md)
+
+---
+
+## 1. Input JSON Schema
+
+O arquivo de entrada deve conter uma lista de artigos sob a chave `articles`.
+
+```json
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "required": ["articles"],
+  "properties": {
+    "articles": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "properties": {
+          "trafilatura": {
+            "type": "object",
+            "properties": {
+              "text": { "type": ["string", "null"] },
+              "error": { "type": ["string", "null"] }
+            }
+          },
+          "newspaper4k": {
+            "type": "object",
+            "properties": {
+              "text": { "type": ["string", "null"] },
+              "error": { "type": ["string", "null"] }
+            }
+          },
+          "readability": {
+            "type": "object",
+            "properties": {
+              "cleaned_text": { "type": ["string", "null"] },
+              "error": { "type": ["string", "null"] }
+            }
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+---
+
+## 2. Output JSON Schema
+
+O arquivo de saída mantém todos os campos, metadados e ordem originais, adicionando obrigatoriamente `selected_extractor`.
+
+```json
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "type": "object",
+  "required": ["articles"],
+  "properties": {
+    "articles": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["selected_extractor"],
+        "properties": {
+          "selected_extractor": {
+            "type": "string",
+            "enum": ["trafilatura", "newspaper4k", "readability"]
+          },
+          "trafilatura": { "type": "object" },
+          "newspaper4k": { "type": "object" },
+          "readability": { "type": "object" }
+        }
+      }
+    }
+  }
+}
+```