- Add ArchiveInput creation in Pipeline.run()
- Import ArchiveInput in pipeline.py
- Add archive_file and skipped to output field recording
- Add ArchiveProcessor to CLI processors list
- Update PipelineConfig steps to include archive
- Added intermediate_path config field with KB_INTERMEDIATE_PATH env var support
- Updated Pipeline to use configurable intermediate directory
- Updated CLI to default to intermediate_path instead of cwd
- Completed requirements: INT-01, INT-02, INT-03
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Import get_settings at top of process_command
- Change default output_dir from cwd to settings.intermediate_path
- Update docstring to document default behavior
- Users can still override with -o flag
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Change default output_dir from cwd to settings.intermediate_path
- Reorder settings initialization before output_dir assignment
- Maintain output_dir parameter for testing/custom override
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add intermediate_path field with default ~/work/intermediate/analysis/
- Follow pydantic-settings naming convention for env var override
- Users can customize via KB_INTERMEDIATE_PATH environment variable
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Update ROADMAP.md to include INT-03 (configurable intermediate path)
- Update 05-01 plan description to reflect INT-03 coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add INT-03 requirement to 05-01-PLAN.md requirements field
- Add Task 1 for KB_INTERMEDIATE_PATH config in KBSettings
- Update Task 2 and Task 3 to use configurable path instead of hardcoded
- Update 05-CONTEXT.md to reflect new decision (config support vs hardcoded)
Addresses checker issue: requirement_coverage for INT-03
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: kb_workflow.py contained 312 lines of deprecated legacy
code that was confusing AI agents. The legacy functions have been
completely replaced by the kb_pipeline package.
Changes:
- Removed all legacy functions (is_video_content, check_duplicate, etc.)
- Kept only process_command() and main() entry points
- Updated SKILL.md to remove old command examples
- File reduced from 495 to 143 lines (71% reduction)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Zhihu HTML structure includes noscript fallback images:
<figure><img data-actualsrc=...><noscript><img src=...></noscript></figure>
After fixing lazy-load (replacing src with data-actualsrc), both imgs
have real URLs, causing duplicates. The fix removes noscript tags before
processing img elements.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add _normalize_metadata() to flatten nested metadata structure
- Fix generate_frontmatter() to use correct metadata fields
- Change _sanitize_filename() to return standardized "bilibili_{bvid}" format
Fixes: analysis.md frontmatter showing "未命名"/0/"未知" instead of actual values
Fixes: audio filename using video title instead of platform_bvid format
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All fixes verified working:
- Video download for videos without official subtitles
- Whisper transcription with correct offset parsing
- Text extraction from Whisper JSON format
- End-to-end analysis with real video content
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Handle Whisper transcription format (array of segments with text)
- Extract full text from transcription array before passing to DeepAnalyzer
- Fix offsets parsing to support both dict and list formats
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add download_video() to BilibiliAdapter using yt-dlp
- Add bilibili_cookie config to Settings
- Update DownloadProcessor to call video download for bilibili videos
- Add idempotency check to ASRProcessor for existing subtitle files
- Fix indentation error in asr.py
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- When content_type is video and no official subtitle available:
- Download video using yt-dlp
- Extract audio for Whisper fallback
- Update content_file with actual video path
This enables the pipeline to handle videos that don't have
official Bilibili subtitles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- When content_type is video and no official subtitle available:
- Download video using yt-dlp
- Extract audio for Whisper fallback
- Update content_file with actual video path
This enables the pipeline to handle videos that don't have
official Bilibili subtitles.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- When a step is skipped (already completed), ask user if they want to re-run
- If skipping, scan content_dir for existing files and populate results
- Support both new naming (platform_id_subtitle.json) and old naming (platform_id.json)
- This fixes the issue where analyze step couldn't find subtitle files
when ASR step was skipped
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add content_dir field to ASRInput schema
- Pass content_dir from pipeline when creating ASRInput
- Modify _try_official_subtitle to use content_dir for output
- Modify _run_whisper_basic to use content_dir for output
- Add validation for empty video_path in Whisper fallback
Root cause: DownloadProcessor sets content_file="" for video types,
causing Path("").parent to resolve to current directory instead of
content directory. Subtitle files were written to wrong location.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: BilibiliExtractor sync wrappers use asyncio.run() internally,
which fails when called from async context in BilibiliAdapter.
Fix: Call async methods directly instead of sync wrappers.
Verified: Human confirmed fix works (bilibili_BV16cNMzpExh)
Parse duration strings like "12:34" or "01:23:45" into seconds.
Previously only integer duration values were handled.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BilibiliAdapter._fetch_metadata_with_retry() now calls the underlying
async methods directly instead of sync wrappers that use asyncio.run().
This prevents "asyncio.run() cannot be called from a running event loop"
error when the adapter is called from within an already-running event loop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Simplify to single-command workflow: kb-workflow process <url>
- Document new CLI parameters and options
- Add architecture overview
- Remove deprecated multi-step manual process
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Archive ROADMAP and REQUIREMENTS to milestones/
- Delete original ROADMAP.md and REQUIREMENTS.md
- Update PROJECT.md with v1.0 validated requirements
- Update STATE.md for milestone complete status
- Update MILESTONES.md with key accomplishments
- Add VERIFICATION.md for phase 04
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add async run() method to Pipeline class
- Execute processors sequentially with state persistence
- Handle step-specific input types (Parse, Download, ASR, Analyze)
- Support skip_completed for resumable workflows
- Return PipelineOutput with results and content_dir
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test full workflow execution from URL to analysis
- Test workflow error handling
- Test state persistence
- Test workflow resumability (skip completed steps)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add process_command function that executes full pipeline
- Add subparser for 'process' command with URL argument
- Add --output-dir and --skip-completed flags
- Add --no-skip flag to re-run all steps
- Maintain backward compatibility with legacy arguments
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Test process --help shows usage
- Test process <url> calls Pipeline.run()
- Test invalid URL returns error
- Test --output-dir flag
- Test --no-skip flag
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
starmen1265