feat(extractor): add Google News headlines extractor with Foxcape headless and URL resolution

- Add standalone CLI script scripts/extract_google_news.py for Google News RSS scraping
- Integrate foxcape in headless mode as primary stealth anti-bot engine
- Implement parallel article URL resolution using googlenewsdecoder and ThreadPoolExecutor
- Support language and regional locale mapping (-l, --lang, --locale)
- Implement real-time progress logging in stderr and --silent flag
- Add unit, integration, and live E2E tests in tests/test_extract_google_news.py
- Add full SpecKit documentation (specs/002-google-news-extractor/)
- Create comprehensive README.md covering both NLP Classifier and Google News Extractor
This commit is contained in:
2026-08-20 11:50:16 -03:00
parent 67cc40f91a
commit 6e3d57619b
59 changed files with 16118 additions and 2160 deletions
+42
View File
@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/">
<channel>
<generator>NFE/5.0</generator>
<title>"inteligencia artificial" - Google Notícias</title>
<link>https://news.google.com/search?q=inteligencia+artificial&amp;hl=pt-BR&amp;gl=BR&amp;ceid=BR:pt-BR</link>
<language>pt-BR</language>
<webMaster>news-webmaster@google.com</webMaster>
<copyright>2026 Google Inc.</copyright>
<lastBuildDate>Thu, 20 Aug 2026 13:00:00 GMT</lastBuildDate>
<description>Google Notícias</description>
<item>
<title>Empresas aceleram adoção de inteligência artificial no Brasil - InfoMoney</title>
<link>https://news.google.com/rss/articles/CBMiYWh0dHBzOi8vd3d3LmluZm9tb25leS5jb20uYnIvbmVnb2Npb3MvYWRvcGNhby1kZS1pYS1uby1icmFzaWwtY3Jlc2NlLTQwLXF1YWRyby1lc3BlY2lhbC1zZXRvci8</link>
<guid isPermaLink="false">CBMiYWh0dHBzOi8vd3d3LmluZm9tb25leS5jb20uYnIvbmVnb2Npb3MvYWRvcGNhby1kZS1pYS1uby1icmFzaWwtY3Jlc2NlLTQwLXF1YWRyby1lc3BlY2lhbC1zZXRvci8</guid>
<pubDate>Thu, 20 Aug 2026 10:30:00 GMT</pubDate>
<description>&lt;ol&gt;&lt;li&gt;&lt;a href="https://news.google.com/rss/articles/CBMiYWh0dHBzOi8vd3d3LmluZm9tb25leS5jb20uYnIvbmVnb2Npb3MvYWRvcGNhby1kZS1pYS1uby1icmFzaWwtY3Jlc2NlLTQwLXF1YWRyby1lc3BlY2lhbC1zZXRvci8" target="_blank"&gt;Empresas aceleram adoção de inteligência artificial no Brasil&lt;/a&gt;&amp;nbsp;&amp;nbsp;&lt;font color="#6f6f6f"&gt;InfoMoney&lt;/font&gt;&lt;p&gt;Levantamento mostra crescimento expressivo na automação e ganhos de produtividade empresarial.&lt;/p&gt;&lt;/li&gt;&lt;/ol&gt;</description>
<source url="https://www.infomoney.com.br">InfoMoney</source>
</item>
<item>
<title>Nova IA generativa promete revolucionar diagnósticos médicos - G1</title>
<link>https://news.google.com/rss/articles/CBMiX2h0dHBzOi8vZzEuZ2xvYm8uY29tL3RlY25vbG9naWEvbm90aWNpYS8yMDI2LzA4LzIwL25vdmEtaWEtZ2VuZXJhdGl2YS1kaWFnbm9zdGljb3MtbWVkaWNvcy5naHRtbA</link>
<guid isPermaLink="false">CBMiX2h0dHBzOi8vZzEuZ2xvYm8uY29tL3RlY25vbG9naWEvbm90aWNpYS8yMDI2LzA4LzIwL25vdmEtaWEtZ2VuZXJhdGl2YS1kaWFnbm9zdGljb3MtbWVkaWNvcy5naHRtbA</guid>
<pubDate>Thu, 20 Aug 2026 09:15:00 GMT</pubDate>
<description>&lt;p&gt;Pesquisadores anunciam modelo capaz de detectar patologias precoces com alta acurácia.&lt;/p&gt;</description>
<source url="https://g1.globo.com">G1</source>
</item>
<item>
<title>IA e o futuro do trabalho: desafios e regulamentações - Folha de S.Paulo</title>
<link>https://news.google.com/rss/articles/CBMiWWh0dHBzOi8vd3d3MS5mb2xoYS51b2wuY29tLmJyL21lcmNhZG8vMjAyNi8wOC9pYS1lLW8tZnV0dXJvLWRvLXRyYWJhbGhvLWRlc2FmaW9zLnNodG1s</link>
<guid isPermaLink="false">CBMiWWh0dHBzOi8vd3d3MS5mb2xoYS51b2wuY29tLmJyL21lcmNhZG8vMjAyNi8wOC9pYS1lLW8tZnV0dXJvLWRvLXRyYWJhbGhvLWRlc2FmaW9zLnNodG1s</guid>
<pubDate>Thu, 20 Aug 2026 08:00:00 GMT</pubDate>
<description>IA e o futuro do trabalho: desafios e regulamentações - Folha de S.Paulo</description>
<source url="https://www1.folha.uol.com.br">Folha de S.Paulo</source>
</item>
<item>
<title>Item sem link</title>
<pubDate>Thu, 20 Aug 2026 07:00:00 GMT</pubDate>
<description>Este item não deve ser retornado pois falta link.</description>
</item>
</channel>
</rss>