Major changes:
- Replace Nitter RSS with Playwright browser automation for direct X scraping
- Scrape all 37 configured tech accounts in parallel
- Add OpenRouter AI integration for topic-based summaries (xiaomi/mimo-v2-flash:free model)
- Update prompts for factual, emotion-free analysis with post links
- Add console output for newsletter preview in dry-run mode
- Update Dockerfile to Playwright v1.57.0 with necessary browser dependencies
- Implement WRAP workflow method for AI-assisted development guidance
Technical improvements:
- Fixed TypeScript compilation (unused parameter in XScraper)
- Newsletter pipeline successfully processes 37 accounts -> AI summaries -> HTML email
- Full end-to-end test validated: scraping, processing, AI generation, email template
Pipeline flow:
1. Scrape X profiles with Playwright (parallel, configurable timeout)
2. Filter tweets by time window and content type
3. Categorize into AI/ML, Software Engineering, Tech & Startups
4. Generate AI summaries for each topic
5. Create cross-topic daily insights
6. Render HTML newsletter with highlights and trending topics
7. Send via email (or print to console in dry-run mode)
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
ksalk
fabfc2b520