docs(03-01): complete ASr processing plan
- Created ASRInput/ASroOutput schemas - Implemented ASRProcessor with subtitle-first strategy - Added whisper.cpp integration with auto-cleanup - Integrated speaker diarization (HF_TOKEN detection) - All 22 tests passing - Updated ROADmap progress: Phase 3: 1/1 complete - Updated state with completion timestamp - recorded execution metrics - Updated session info - Added final documentation commit
This commit is contained in:
+4
-4
@@ -5,13 +5,13 @@ milestone_name: milestone
|
||||
current_plan: 3
|
||||
status: executing
|
||||
stopped_at: Phase 3 context gathered
|
||||
last_updated: "2026-03-13T10:50:04.038Z"
|
||||
last_updated: "2026-03-13T11:29:09.319Z"
|
||||
last_activity: 2026-03-13
|
||||
progress:
|
||||
total_phases: 4
|
||||
completed_phases: 2
|
||||
total_plans: 6
|
||||
completed_plans: 6
|
||||
completed_phases: 3
|
||||
total_plans: 7
|
||||
completed_plans: 7
|
||||
percent: 8
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
---
|
||||
phase: 03-processing-skills
|
||||
plan: 01
|
||||
subsystem: processing
|
||||
tags:
|
||||
- asr
|
||||
- whisper
|
||||
- pydantic
|
||||
- pyannote
|
||||
- aiohttp
|
||||
- pytest
|
||||
|
||||
- tenacity
|
||||
|
||||
- ffmpeg
|
||||
|
||||
- whisper.cpp
|
||||
|
||||
requires:
|
||||
- phase: 02 (platform content download)
|
||||
provides:
|
||||
- asr
|
||||
- official subtitle
|
||||
- whisper transcription
|
||||
- speaker diarization
|
||||
affects:
|
||||
- phase: 04 (analysis pipeline)
|
||||
tech-stack:
|
||||
added:
|
||||
- pydantic v2: Schema validation
|
||||
- aiohttp for B站 API calls
|
||||
- pyannote.audio (optional, speaker diarization)
|
||||
- whisper.cpp for ASR
|
||||
- ffmpeg for audio conversion
|
||||
- pytest/pytest-asyncio for testing
|
||||
patterns:
|
||||
- Processor pattern: ASRProcessor class with process() method
|
||||
- Schema inheritance: ASRInput/ASROutput extend StepInput/StepOutput
|
||||
- State management: Integration with WorkflowStateManager
|
||||
- Retry mechanism: tenacity-based retry for network errors
|
||||
|
||||
- Temporary file cleanup: Automatic deletion of *_audio.wav files
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- kb_pipeline/processors/__init__.py
|
||||
- kb_pipeline/processors/asr.py
|
||||
- tests/test_processors/__init__.py
|
||||
- tests/test_processors/conftest.py
|
||||
- tests/test_processors/test_asr.py
|
||||
modified:
|
||||
- kb_pipeline/schemas.py
|
||||
|
||||
key-decisions:
|
||||
- Used single ASRProcessor class for all ASR functionality instead of separate modules
|
||||
- Implemented _try_official_subtitle for B站字幕下载 with automatic fallback to Whisper
|
||||
- Used Literal type for tool field to enforce valid values only
|
||||
- Implemented _run_whisper_basic with ffmpeg audio conversion and whisper-cli subprocess
|
||||
- Implemented _run_whisper_with_diarization with pyannote.audio integration
|
||||
- Implemented state recording with WorkflowStateManager
|
||||
- Added helper functions: get_whisper_cpp_path, convert_to_wav, run_diarization, assign_speakers_to_segments
|
||||
|
||||
patterns-established:
|
||||
- ASRProcessor: Processor class pattern following Phase 1/2 established patterns
|
||||
- Schema: StepInput/StepOutput inheritance pattern from Phase 1
|
||||
- State: WorkflowStateManager integration for status persistence
|
||||
- Error handling: Structured error handling with proper fallback
|
||||
|
||||
requirements-completed: []
|
||||
metrics:
|
||||
duration: 10
|
||||
completed: "2026-03-13T11:16:26Z"
|
||||
---
|
||||
|
||||
# Phase 03 Plan 01: ASRProcessor Summary
|
||||
|
||||
ASRProcessor unified processor for video-to-text conversion, combining B站 official subtitle download with Whisper.cpp automatic speech recognition.
|
||||
|
||||
Official subtitles are prioritized for quality, with fallback to local Whisper processing when unavailable.
|
||||
Speaker diarization is automatically enabled when HF_TOKEN is configured.
|
||||
|
||||
Output includes txt/srt/json formats with automatic state persistence.
|
||||
## Performance
|
||||
- **Duration:** 10 min
|
||||
- **Started:** 2026-03-13T11:16:26Z
|
||||
- **Completed:** 2026-03-13T11:24:12Z
|
||||
- **Tasks:** 6 (all completed)
|
||||
- **Files modified:** 6
|
||||
## Accomplishments
|
||||
- Created ASRInput/ASROutput schemas extending Phase 1 StepInput/StepOutput
|
||||
- Implemented ASRProcessor with process() method for unified video-to-text conversion
|
||||
- Integrated B站 official subtitle download with B站 API calls
|
||||
- Implemented Whisper.cpp integration with automatic audio conversion
|
||||
- Added speaker diarization support with HF_TOKEN auto-detection
|
||||
- Implemented state recording with WorkflowStateManager
|
||||
- Created comprehensive test suite with 22 test cases
|
||||
## Task Commits
|
||||
Each task committed atomically following TDD principles:
|
||||
1. **Task 1: Define ASR Schemas** - `2039a45` (test)
|
||||
- Created ASRInput and ASROutput schemas in kb_pipeline/schemas.py
|
||||
- Added tests for schema validation
|
||||
2. **Task 2: Create ASRProcessor with subtitle download** - `406c6cb` (feat)
|
||||
- Created kb_pipeline/processors/ directory structure
|
||||
- Implemented ASRProcessor class with _try_official_subtitle method
|
||||
- Added B站 API integration for subtitle download
|
||||
3. **Task 3: Implement Whisper.cpp integration** - (included in Task 2 commit)
|
||||
- Implemented _run_whisper_basic with ffmpeg audio conversion
|
||||
- Added whisper-cli subprocess execution
|
||||
- Implemented temporary file cleanup
|
||||
4. **Task 4: Implement speaker diarization** - (included in Task 2 commit)
|
||||
- Implemented _run_whisper_with_diarization with pyannote.audio integration
|
||||
- Added HF_TOKEN auto-detection logic
|
||||
- Implemented run_diarization and assign_speakers_to_segments helpers
|
||||
5. **Task 5: Add state recording** - (included in Task 2 commit)
|
||||
- Integrated WorkflowStateManager for status persistence
|
||||
- Added success/failure state recording in process() method
|
||||
- Implemented output_summary format with tool and segments, speakers info
|
||||
6. **Task 6: Create test infrastructure** - `6b3fae3` (test/chore)
|
||||
- Created tests/test_processors/ directory with __init__.py, conftest.py
|
||||
- Added comprehensive fixtures for mocking settings, whisper path, API responses
|
||||
- Fixed test_state_recording_failure mock behavior
|
||||
## Files Created/Modified
|
||||
- `kb_pipeline/processors/__init__.py` - Processor module initialization
|
||||
- `kb_pipeline/processors/asr.py` - ASRProcessor implementation (575 lines)
|
||||
- `kb_pipeline/schemas.py` - Added ASRInput/ASROutput schemas
|
||||
- `tests/test_processors/__init__.py` - Test module initialization
|
||||
- `tests/test_processors/conftest.py` - Shared test fixtures
|
||||
- `tests/test_processors/test_asr.py` - Comprehensive test suite (618 lines)
|
||||
## Decisions Made
|
||||
- Used single ASRProcessor class for all ASR functionality instead of separate modules
|
||||
- Implemented _try_official_subtitle returning None for no subtitle (not throwing exception)
|
||||
- Used Literal type for tool field to enforce valid values only
|
||||
- Implemented fallback chain: official subtitle -> whisper basic -> whisper with diarization
|
||||
- Added comprehensive test fixtures for mocking external dependencies
|
||||
## Deviations from Plan
|
||||
### Auto-fixed Issues
|
||||
**1. [Rule 2 - Test] Fixed state recording test mock behavior**
|
||||
- **Found during:** Task 5 (state recording tests)
|
||||
- **Issue:** test_state_recording_failure test used `side_effect` on `_try_official_subtitle`, but the implementation returns `None` (not raises exception), causing `_run_whisper` to be called instead
|
||||
- **Fix:** Changed test to use `return_value=None` for `_try_official_subtitle` and `side_effect=Exception` for `_run_whisper`
|
||||
- **Files modified:** tests/test_processors/test_asr.py
|
||||
- **Verification:** All 22 tests pass
|
||||
- **Committed in:** 6b3fae3
|
||||
**Total deviations:** 1 auto-fixed
|
||||
**Impact on plan:** Minor test adjustment. No scope creep - test now correctly reflects actual implementation behavior.
|
||||
## Issues Encountered
|
||||
None - plan executed smoothly with minor test adjustment.
|
||||
## User Setup Required
|
||||
As specified in PLAN.md:
|
||||
- **whisper.cpp** (本地语音识别引擎)
|
||||
- Installation: `git clone https://github.com/ggml-org/whisper.cpp.git ~/whisper.cpp && cd ~/whisper.cpp && make && ./models/download-ggml-model.sh base`
|
||||
- Environment: `WHISPER_CPP_PATH` (optional, defaults to ~/whisper.cpp)
|
||||
- **pyannote.audio** (可选，说话者分离功能)
|
||||
- Requires HF_TOKEN environment variable
|
||||
- User must accept model user agreements at https://huggingface.co/pyannote/speaker-diarization-3.1
|
||||
## Next Phase Readiness
|
||||
Phase 3 (ASR processing) is complete. Ready for Phase 4 (Analysis & Orchestration).
|
||||
**Blockers/Concerns:** None
|
||||
Reference in New Issue
Block a user