Change Segment(index=1, ...) to Segment(id=1, ...) to match
the actual Segment class definition in parser.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add enable_cache parameter to SRTProcessor.__init__()
- Create SemanticCache instance when semantic segmentation is enabled
- Pass cache to UnifiedSegmentationStrategy through SemanticSegmentationEngine
- Add logging for cache status (enabled/disabled)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update CORRECTION_PROMPT to explicitly require adding punctuation
- Fix missing punctuation in thunderstorm and lightning sections
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SimpleAIClient: Add disable_reasoning parameter to control AI thinking mode
- ASRCorrector: Use disable_reasoning=True to get direct correction results
- ArchiveProcessor: Extract title from frontmatter and include in filename
File naming format: {YYYYMMDD}_{content_id}_{title}_analysis.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add logging warning in DeepAnalyzer.__init__
- Warn that Bilibili video should use SRTProcessor instead
- Keep functionality intact for Zhihu and other platforms
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Import SRTProcessor from processors module
- Add SRTProcessor to processors list
- Update PipelineConfig.steps to include 'srt' step
- Pipeline will select SRTProcessor or AnalyzeProcessor based on platform
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Bilibili video detection: platform=bilibili and content_type=video
- Skip analyze step for Bilibili videos, use srt step instead
- Skip srt step for non-Bilibili-video content
- Add SRTInput import and step input creation logic
- Update _populate_skipped_step_results for srt step
- Add output_file, paragraphs, tags, corrected to result fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add SemanticSegmenter and ASRCorrector to __all__
- Maintain lazy loading for optional dependencies
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add SRTInput/SRTOutput schemas to pipeline
- Create SRTProcessor following BaseProcessor pattern
- Generate Markdown with YAML frontmatter
- Support metadata extraction and formatting
- Handle missing metadata gracefully
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use LLM to correct obvious ASR errors (homophones, missing words)
- Batch processing for long text (5000 chars per batch)
- Fallback to original text when AI unavailable
- Simple sentence-based text splitting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge short segments into paragraphs
- Split on time gaps (>3 seconds by default)
- Enforce max paragraph length (500 chars by default)
- Simple rule-based approach (no LLM needed)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Segment dataclass for subtitle segments
- Add parse_srt() function to parse SRT content string
- Add parse_srt_file() function to parse from file path
- Support standard SRT format with timestamps
- Handle special characters and multi-line subtitles
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add StealthConfig Pydantic model with all playwright-stealth options
- Add apply_stealth_sync/apply_stealth_async functions for browser context
- Add get_default_stealth_config convenience function
- Add 9 unit tests covering config defaults, stealth application, and disabled mode
- Update conftest.py with stealth_config and mock_browser_context fixtures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add proxy_pool parameter to ZhihuCrawler constructor
- Add _current_proxy tracking for health check feedback
- Implement _parse_proxy_url static method for URL parsing
- Implement _on_request_success/failure for proxy health feedback
- Prioritize proxy_pool over config.proxy for multi-proxy support
- Maintain backward compatibility with single proxy config
Tests:
- test_crawler_accepts_proxy_pool
- test_crawler_without_proxy_pool (backward compatibility)
- test_parse_proxy_url (HTTP, auth, SOCKS5)
- test_proxy_pool_takes_priority_over_config
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename kb_workflow.py to cli.py to avoid conflict with kb_workflow/ package.
Add __main__.py to support `python -m kb_workflow` execution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace ZhihuPlaywrightExtractor with ZhihuCrawler
- Add comment crawling with include_comments=True by default
- Implement cookie expiration detection and QR code login trigger
- Add idempotency check via IdempotencyChecker
- Add comment formatting with "## 热门评论" section
- Convert HTML content to Markdown using html2text
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CrawlerConfig with ProxyConfig, delay, and retry settings
- Integrate proxy configuration via _start_browser_with_proxy()
- Add random delay via _apply_random_delay()
- Add retry logic via _crawl_with_retry using tenacity
- Add ZhihuApiInterceptor tests for x-zse-96 handling
- All tests pass
- Add CrawlerConfig with ProxyConfig, delay, and retry settings
- Support HTTP_PROXY/HTTPS_PROXY environment variable parsing
- Add proxy_required validation with clear error message
- Add get_crawler_config singleton for configuration access
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add COMMENT_API_TEMPLATE for comment API URL
- Implement crawl_comments() returning CommentsResult
- Use order=hot for popular comments sorting
- Only fetch first-level comments (no nested replies)
- Add include_comments parameter to crawl_answer()
- Export CommentAuthor, CommentData, CommentsResult in __init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CommentAuthor model for comment author info
- Add CommentData model with content, author, likes, child_count
- Add CommentsResult model for crawl operation result
- Update CrawlResult to include optional comments field
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add ZhihuCrawler class with crawl_answer() and crawl_article() methods
- Implement ZhihuApiInterceptor for capturing /api/v4/ responses
- Add _parse_author(), _parse_metadata(), _parse_answer_data(), _parse_article_data() helpers
- Support DOM parsing fallback for zhuanlan articles
- Export ZhihuCrawler and ZhihuApiInterceptor from __init__.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Pydantic models: AnswerData, ArticleData, ContentMetadata, AuthorInfo, CrawlResult
- Create test fixtures in conftest.py with mock browser and sample API responses
- Set up crawler module with __init__.py exports
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add login_command function with QR code login, cookie import, and status check
- Add login subcommand with --import-cookie and --status options
- Update CLI help text with login examples
- Fix mock paths in tests to match actual import locations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CookieValidationResult Pydantic model with valid, username, user_id, reason fields
- Add validate_cookie async method to ZhihuLogin class
- Validate cookie via Zhihu /api/v4/me endpoint
- Handle 200, 401, and other status codes appropriately
- Handle network errors gracefully
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TDD GREEN phase - implementation passes all 8 tests:
- ZhihuLoginResult: Pydantic model for login results
- ZhihuLogin: Main class with QR code login flow
- LOGIN_URL: https://www.zhihu.com/signin
- CANVAS_SELECTORS: Multiple fallback selectors for QR canvas
- _find_qrcode_canvas(): Find canvas with fallback selectors
- _save_qrcode_image(): Save QR code to intermediate directory
- check_login_cookie(): Detect z_c0 cookie for login state
- get_cookies_str(): Extract cookie string from context
- _wait_for_login(): Poll with configurable timeout (tenacity)
- login_by_qrcode(): Main entry point for login flow
Key design decisions:
- Uses tenacity Retrying for configurable timeout
- QR code saved to intermediate_path with timestamp
- Multiple canvas selectors for robustness
- Proper browser lifecycle management
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CookieManager saves cookies to ~/.kb_env file
- Updates existing ZHIHU_COOKIE line or appends new
- Preserves other configuration lines
- Sets file permissions to 600 for security
- Loads cookies from env file
- Clears get_settings cache after saving
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Parse Chrome Cookie Editor JSON format
- Filter cookies by zhihu.com domain
- Handle malformed JSON with ValueError
- Handle file not found with ValueError
- Return cookie string in "name=value; name2=value2" format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test parse_json_cookies for Chrome Cookie Editor format
- Test domain filtering for zhihu.com
- Test empty array handling
- Test malformed JSON error handling
- Test file not found error handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move kb-deep-analyzer/deep_analyzer.py to kb_workflow/deep_analyzer/__init__.py
- Update analyze.py to import from kb_workflow.deep_analyzer directly
- Remove sys.path manipulation for external package
- DeepAnalyzer, SimpleAIClient, generate_frontmatter now available as:
from kb_workflow.deep_analyzer import DeepAnalyzer, SimpleAIClient, generate_frontmatter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add one more .parent to reach correct directory level
- Fixed path: /home/.../kb-skills/kb-deep-analyzer (was /home/.../kb-skills/kb_workflow/kb-deep-analyzer)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
starmen1265