feat(extractor): add Google News headlines extractor with Foxcape headless and URL resolution

Browse Source

- Add standalone CLI script scripts/extract_google_news.py for Google News RSS scraping
- Integrate foxcape in headless mode as primary stealth anti-bot engine
- Implement parallel article URL resolution using googlenewsdecoder and ThreadPoolExecutor
- Support language and regional locale mapping (-l, --lang, --locale)
- Implement real-time progress logging in stderr and --silent flag
- Add unit, integration, and live E2E tests in tests/test_extract_google_news.py
- Add full SpecKit documentation (specs/002-google-news-extractor/)
- Create comprehensive README.md covering both NLP Classifier and Google News Extractor

...

This commit is contained in:

andreferraro

2026-08-20 11:50:16 -03:00

parent 67cc40f91a

commit 6e3d57619b

59 changed files with 16118 additions and 2160 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

graphify-out/graph.html

+4 -4

File diff suppressed because one or more lines are too long