- Add CrawlerConfig with ProxyConfig, delay, and retry settings
- Support HTTP_PROXY/HTTPS_PROXY environment variable parsing
- Add proxy_required validation with clear error message
- Add get_crawler_config singleton for configuration access
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add COMMENT_API_TEMPLATE for comment API URL
- Implement crawl_comments() returning CommentsResult
- Use order=hot for popular comments sorting
- Only fetch first-level comments (no nested replies)
- Add include_comments parameter to crawl_answer()
- Export CommentAuthor, CommentData, CommentsResult in __init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test crawl_comments returns CommentsResult
- Test comments use hot sorting (order=hot)
- Test comment has required fields (content, author, like_count)
- Test no nested replies (first-level only)
- Test include_comments parameter control
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add test_answer_crawler.py with tests for crawl_answer()
- Add test_article_crawler.py with tests for crawl_article()
- Add test_metadata_extraction.py with tests for author/metadata parsing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Pydantic models: AnswerData, ArticleData, ContentMetadata, AuthorInfo, CrawlResult
- Create test fixtures in conftest.py with mock browser and sample API responses
- Set up crawler module with __init__.py exports
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
starmen1265