- SimpleAIClient: Add disable_reasoning parameter to control AI thinking mode
- ASRCorrector: Use disable_reasoning=True to get direct correction results
- ArchiveProcessor: Extract title from frontmatter and include in filename
File naming format: {YYYYMMDD}_{content_id}_{title}_analysis.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add logging warning in DeepAnalyzer.__init__
- Warn that Bilibili video should use SRTProcessor instead
- Keep functionality intact for Zhihu and other platforms
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test Bilibili video uses SRTProcessor (not AnalyzeProcessor)
- Test Bilibili article uses AnalyzeProcessor (not SRTProcessor)
- Test Zhihu answer uses AnalyzeProcessor
- Test Markdown output format verification
All 4 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Import SRTProcessor from processors module
- Add SRTProcessor to processors list
- Update PipelineConfig.steps to include 'srt' step
- Pipeline will select SRTProcessor or AnalyzeProcessor based on platform
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Bilibili video detection: platform=bilibili and content_type=video
- Skip analyze step for Bilibili videos, use srt step instead
- Skip srt step for non-Bilibili-video content
- Add SRTInput import and step input creation logic
- Update _populate_skipped_step_results for srt step
- Add output_file, paragraphs, tags, corrected to result fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Create 12-01-SUMMARY.md with implementation details
- Update STATE.md with current position (Phase 12, 50% complete)
- Update ROADMAP.md with plan progress
- Mark requirements SRT-01~04, OUT-01~02, OUT-04 as complete
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add SemanticSegmenter and ASRCorrector to __all__
- Maintain lazy loading for optional dependencies
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add SRTInput/SRTOutput schemas to pipeline
- Create SRTProcessor following BaseProcessor pattern
- Generate Markdown with YAML frontmatter
- Support metadata extraction and formatting
- Handle missing metadata gracefully
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use LLM to correct obvious ASR errors (homophones, missing words)
- Batch processing for long text (5000 chars per batch)
- Fallback to original text when AI unavailable
- Simple sentence-based text splitting
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test obvious homophone error correction
- Test contextual error correction
- Test no overcorrection on correct text
- Test AI unavailable fallback
- Test batch processing for long text
- Test empty text handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Merge short segments into paragraphs
- Split on time gaps (>3 seconds by default)
- Enforce max paragraph length (500 chars by default)
- Simple rule-based approach (no LLM needed)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test merging short segments into paragraphs
- Test segmentation on topic change (time gap)
- Test max paragraph length constraint
- Test empty segments handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Segment dataclass for subtitle segments
- Add parse_srt() function to parse SRT content string
- Add parse_srt_file() function to parse from file path
- Support standard SRT format with timestamps
- Handle special characters and multi-line subtitles
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test standard SRT format parsing
- Test special character handling
- Test error handling for invalid/empty content
- Test file parsing functionality
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete .planning/REQUIREMENTS.md (archived to milestones/)
- Delete .planning/phases/11-stealth-fingerprinting/* (moved to milestones/v1.4-phases/)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test crawler accepts stealth_config parameter
- Test default stealth enabled behavior
- Test stealth can be disabled
- Test _start_browser_with_proxy calls apply_stealth_sync
- Test launch args include --disable-blink-features
- Test backward compatibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add StealthConfig Pydantic model with all playwright-stealth options
- Add apply_stealth_sync/apply_stealth_async functions for browser context
- Add get_default_stealth_config convenience function
- Add 9 unit tests covering config defaults, stealth application, and disabled mode
- Update conftest.py with stealth_config and mock_browser_context fixtures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add architecture research for proxy pool, behavior camouflage,
captcha detection, and error recovery
- Update features, pitfalls, and stack documentation
- Add phase 10 validation checklist
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add proxy_pool parameter to ZhihuCrawler constructor
- Add _current_proxy tracking for health check feedback
- Implement _parse_proxy_url static method for URL parsing
- Implement _on_request_success/failure for proxy health feedback
- Prioritize proxy_pool over config.proxy for multi-proxy support
- Maintain backward compatibility with single proxy config
Tests:
- test_crawler_accepts_proxy_pool
- test_crawler_without_proxy_pool (backward compatibility)
- test_parse_proxy_url (HTTP, auth, SOCKS5)
- test_proxy_pool_takes_priority_over_config
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
starmen1265