- Add StealthConfig Pydantic model with all playwright-stealth options
- Add apply_stealth_sync/apply_stealth_async functions for browser context
- Add get_default_stealth_config convenience function
- Add 9 unit tests covering config defaults, stealth application, and disabled mode
- Update conftest.py with stealth_config and mock_browser_context fixtures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add proxy_pool parameter to ZhihuCrawler constructor
- Add _current_proxy tracking for health check feedback
- Implement _parse_proxy_url static method for URL parsing
- Implement _on_request_success/failure for proxy health feedback
- Prioritize proxy_pool over config.proxy for multi-proxy support
- Maintain backward compatibility with single proxy config
Tests:
- test_crawler_accepts_proxy_pool
- test_crawler_without_proxy_pool (backward compatibility)
- test_parse_proxy_url (HTTP, auth, SOCKS5)
- test_proxy_pool_takes_priority_over_config
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename kb_workflow.py to cli.py to avoid conflict with kb_workflow/ package.
Add __main__.py to support `python -m kb_workflow` execution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace ZhihuPlaywrightExtractor with ZhihuCrawler
- Add comment crawling with include_comments=True by default
- Implement cookie expiration detection and QR code login trigger
- Add idempotency check via IdempotencyChecker
- Add comment formatting with "## 热门评论" section
- Convert HTML content to Markdown using html2text
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CrawlerConfig with ProxyConfig, delay, and retry settings
- Integrate proxy configuration via _start_browser_with_proxy()
- Add random delay via _apply_random_delay()
- Add retry logic via _crawl_with_retry using tenacity
- Add ZhihuApiInterceptor tests for x-zse-96 handling
- All tests pass
- Add CrawlerConfig with ProxyConfig, delay, and retry settings
- Support HTTP_PROXY/HTTPS_PROXY environment variable parsing
- Add proxy_required validation with clear error message
- Add get_crawler_config singleton for configuration access
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add COMMENT_API_TEMPLATE for comment API URL
- Implement crawl_comments() returning CommentsResult
- Use order=hot for popular comments sorting
- Only fetch first-level comments (no nested replies)
- Add include_comments parameter to crawl_answer()
- Export CommentAuthor, CommentData, CommentsResult in __init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CommentAuthor model for comment author info
- Add CommentData model with content, author, likes, child_count
- Add CommentsResult model for crawl operation result
- Update CrawlResult to include optional comments field
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ZhihuCrawler class with crawl_answer() and crawl_article() methods
- Implement ZhihuApiInterceptor for capturing /api/v4/ responses
- Add _parse_author(), _parse_metadata(), _parse_answer_data(), _parse_article_data() helpers
- Support DOM parsing fallback for zhuanlan articles
- Export ZhihuCrawler and ZhihuApiInterceptor from __init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Pydantic models: AnswerData, ArticleData, ContentMetadata, AuthorInfo, CrawlResult
- Create test fixtures in conftest.py with mock browser and sample API responses
- Set up crawler module with __init__.py exports
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add login_command function with QR code login, cookie import, and status check
- Add login subcommand with --import-cookie and --status options
- Update CLI help text with login examples
- Fix mock paths in tests to match actual import locations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CookieValidationResult Pydantic model with valid, username, user_id, reason fields
- Add validate_cookie async method to ZhihuLogin class
- Validate cookie via Zhihu /api/v4/me endpoint
- Handle 200, 401, and other status codes appropriately
- Handle network errors gracefully
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TDD GREEN phase - implementation passes all 8 tests:
- ZhihuLoginResult: Pydantic model for login results
- ZhihuLogin: Main class with QR code login flow
- LOGIN_URL: https://www.zhihu.com/signin
- CANVAS_SELECTORS: Multiple fallback selectors for QR canvas
- _find_qrcode_canvas(): Find canvas with fallback selectors
- _save_qrcode_image(): Save QR code to intermediate directory
- check_login_cookie(): Detect z_c0 cookie for login state
- get_cookies_str(): Extract cookie string from context
- _wait_for_login(): Poll with configurable timeout (tenacity)
- login_by_qrcode(): Main entry point for login flow
Key design decisions:
- Uses tenacity Retrying for configurable timeout
- QR code saved to intermediate_path with timestamp
- Multiple canvas selectors for robustness
- Proper browser lifecycle management
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CookieManager saves cookies to ~/.kb_env file
- Updates existing ZHIHU_COOKIE line or appends new
- Preserves other configuration lines
- Sets file permissions to 600 for security
- Loads cookies from env file
- Clears get_settings cache after saving
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Parse Chrome Cookie Editor JSON format
- Filter cookies by zhihu.com domain
- Handle malformed JSON with ValueError
- Handle file not found with ValueError
- Return cookie string in "name=value; name2=value2" format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test parse_json_cookies for Chrome Cookie Editor format
- Test domain filtering for zhihu.com
- Test empty array handling
- Test malformed JSON error handling
- Test file not found error handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move kb-deep-analyzer/deep_analyzer.py to kb_workflow/deep_analyzer/__init__.py
- Update analyze.py to import from kb_workflow.deep_analyzer directly
- Remove sys.path manipulation for external package
- DeepAnalyzer, SimpleAIClient, generate_frontmatter now available as:
from kb_workflow.deep_analyzer import DeepAnalyzer, SimpleAIClient, generate_frontmatter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add one more .parent to reach correct directory level
- Fixed path: /home/.../kb-skills/kb-deep-analyzer (was /home/.../kb-skills/kb_workflow/kb-deep-analyzer)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Extract archive_file from pipeline results and include in response
- Allows callers to get the final archive path instead of only content_dir
- Fixes report display issue where content directory showed working dir instead of archive location
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix download processor to return actual author name (e.g. "HOLA小测佬")
instead of B站用户ID (mid, e.g. "406636263")
- Add 5 test cases for author extraction logic
- Resolves report display issue where UP主名称 showed numeric ID
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Pipeline import from kb_workflow.pipeline.pipeline
- Add Pipeline to __all__ exports list
- Enables: from kb_workflow.pipeline import Pipeline
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove top-level sys.path.insert for kb-deep-analyzer
- Add module-level flag _deep_analyzer_path_added for single execution
- Move sys.path manipulation inside _run_analysis method
- External dependency import now happens lazily at runtime
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unnecessary sys.path.insert for current directory
- Use relative import for zhihu_extractor
- Package imports work correctly without path manipulation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unnecessary sys.path.insert for current directory
- Use pure relative import for markdown_converter
- Package imports work correctly without path manipulation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unnecessary sys.path.insert for project root
- Update docstring to reference kb_workflow.pipeline
- Package imports work correctly without path manipulation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove sys.path manipulation from kb_workflow.py
- Update CLI imports to use kb_workflow.pipeline prefix
- Update all test file imports and patch paths
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move all processor implementations (parse, download, asr, analyze, archive)
- Move all platform adapters (base, bilibili, zhihu)
- Update all internal imports to use kb_workflow.pipeline prefix
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove kb-bilibili-metadata/ and kb-zhihu-metadata/ directories
- Create kb_workflow/__init__.py to export main and process_command
- Update test_cli/test_process_command.py imports for new package structure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Rename kb-workflow to kb_workflow for Python package compatibility
- Move bilibili_extractor.py and bilibili_metadata.py to kb_workflow/metadata/
- Create metadata/__init__.py with BilibiliExporter export
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
starmen1265