Archived

This repository has been archived on 2026-08-07. You can view files and clone it. You cannot open issues or pull requests or push a commit.

Files

T

starmen1265 d8f4df9149 docs(03-01): complete ASr processing plan

- Created ASRInput/ASroOutput schemas
- Implemented ASRProcessor with subtitle-first strategy
- Added whisper.cpp integration with auto-cleanup
- Integrated speaker diarization (HF_TOKEN detection)
- All 22 tests passing

- Updated ROADmap progress: Phase 3: 1/1 complete
- Updated state with completion timestamp

- recorded execution metrics

- Updated session info
- Added final documentation commit

2026-03-13 19:29:27 +08:00

7.1 KiB

Raw Blame History

phase: 03-processing-skills plan: 01 subsystem: processing tags:

asr
whisper
pydantic
pyannote
aiohttp
pytest
tenacity
ffmpeg
whisper.cpp

requires:

phase: 02 (platform content download) provides:
asr
official subtitle
whisper transcription
speaker diarization affects:
phase: 04 (analysis pipeline) tech-stack: added:
- pydantic v2: Schema validation
- aiohttp for B站 API calls
- pyannote.audio (optional, speaker diarization)
- whisper.cpp for ASR
- ffmpeg for audio conversion
pytest/pytest-asyncio for testing patterns:
- Processor pattern: ASRProcessor class with process() method
- Schema inheritance: ASRInput/ASROutput extend StepInput/StepOutput
- State management: Integration with WorkflowStateManager
- Retry mechanism: tenacity-based retry for network errors
- Temporary file cleanup: Automatic deletion of *_audio.wav files

key-files: created: - kb_pipeline/processors/__init__.py - kb_pipeline/processors/asr.py - tests/test_processors/__init__.py - tests/test_processors/conftest.py - tests/test_processors/test_asr.py modified: - kb_pipeline/schemas.py

key-decisions:

Used single ASRProcessor class for all ASR functionality instead of separate modules
Implemented _try_official_subtitle for B站字幕下载 with automatic fallback to Whisper
Used Literal type for tool field to enforce valid values only
Implemented _run_whisper_basic with ffmpeg audio conversion and whisper-cli subprocess
Implemented _run_whisper_with_diarization with pyannote.audio integration
Implemented state recording with WorkflowStateManager
Added helper functions: get_whisper_cpp_path, convert_to_wav, run_diarization, assign_speakers_to_segments

patterns-established:

ASRProcessor: Processor class pattern following Phase 1/2 established patterns
Schema: StepInput/StepOutput inheritance pattern from Phase 1
State: WorkflowStateManager integration for status persistence
Error handling: Structured error handling with proper fallback

requirements-completed: [] metrics: duration: 10 completed: "2026-03-13T11:16:26Z"

Phase 03 Plan 01: ASRProcessor Summary

ASRProcessor unified processor for video-to-text conversion, combining B站 official subtitle download with Whisper.cpp automatic speech recognition.

Official subtitles are prioritized for quality, with fallback to local Whisper processing when unavailable. Speaker diarization is automatically enabled when HF_TOKEN is configured.

Output includes txt/srt/json formats with automatic state persistence.

Performance

Duration: 10 min
Started: 2026-03-13T11:16:26Z
Completed: 2026-03-13T11:24:12Z
Tasks: 6 (all completed)
Files modified: 6

Accomplishments

Created ASRInput/ASROutput schemas extending Phase 1 StepInput/StepOutput
Implemented ASRProcessor with process() method for unified video-to-text conversion
Integrated B站 official subtitle download with B站 API calls
Implemented Whisper.cpp integration with automatic audio conversion
Added speaker diarization support with HF_TOKEN auto-detection
Implemented state recording with WorkflowStateManager
Created comprehensive test suite with 22 test cases

Task Commits

Each task committed atomically following TDD principles:

Task 1: Define ASR Schemas - 2039a45 (test)
- Created ASRInput and ASROutput schemas in kb_pipeline/schemas.py
- Added tests for schema validation
Task 2: Create ASRProcessor with subtitle download - 406c6cb (feat)
- Created kb_pipeline/processors/ directory structure
- Implemented ASRProcessor class with _try_official_subtitle method
- Added B站 API integration for subtitle download
Task 3: Implement Whisper.cpp integration - (included in Task 2 commit)
- Implemented _run_whisper_basic with ffmpeg audio conversion
- Added whisper-cli subprocess execution
- Implemented temporary file cleanup
Task 4: Implement speaker diarization - (included in Task 2 commit)
- Implemented _run_whisper_with_diarization with pyannote.audio integration
- Added HF_TOKEN auto-detection logic
- Implemented run_diarization and assign_speakers_to_segments helpers
Task 5: Add state recording - (included in Task 2 commit)
- Integrated WorkflowStateManager for status persistence
- Added success/failure state recording in process() method
- Implemented output_summary format with tool and segments, speakers info
Task 6: Create test infrastructure - 6b3fae3 (test/chore)
- Created tests/test_processors/ directory with __init__.py, conftest.py
- Added comprehensive fixtures for mocking settings, whisper path, API responses
- Fixed test_state_recording_failure mock behavior

Files Created/Modified

kb_pipeline/processors/__init__.py - Processor module initialization
kb_pipeline/processors/asr.py - ASRProcessor implementation (575 lines)
kb_pipeline/schemas.py - Added ASRInput/ASROutput schemas
tests/test_processors/__init__.py - Test module initialization
tests/test_processors/conftest.py - Shared test fixtures
tests/test_processors/test_asr.py - Comprehensive test suite (618 lines)

Decisions Made

Used single ASRProcessor class for all ASR functionality instead of separate modules
Implemented _try_official_subtitle returning None for no subtitle (not throwing exception)
Used Literal type for tool field to enforce valid values only
Implemented fallback chain: official subtitle -> whisper basic -> whisper with diarization
Added comprehensive test fixtures for mocking external dependencies

Deviations from Plan

Auto-fixed Issues

1. [Rule 2 - Test] Fixed state recording test mock behavior

Found during: Task 5 (state recording tests)
Issue: test_state_recording_failure test used side_effect on _try_official_subtitle, but the implementation returns None (not raises exception), causing _run_whisper to be called instead
Fix: Changed test to use return_value=None for _try_official_subtitle and side_effect=Exception for _run_whisper
Files modified: tests/test_processors/test_asr.py
Verification: All 22 tests pass
Committed in: 6b3fae3 Total deviations: 1 auto-fixed Impact on plan: Minor test adjustment. No scope creep - test now correctly reflects actual implementation behavior.

Issues Encountered

None - plan executed smoothly with minor test adjustment.

User Setup Required

As specified in PLAN.md:

whisper.cpp (本地语音识别引擎)
- Installation: git clone https://github.com/ggml-org/whisper.cpp.git ~/whisper.cpp && cd ~/whisper.cpp && make && ./models/download-ggml-model.sh base
Environment: WHISPER_CPP_PATH (optional, defaults to ~/whisper.cpp)
pyannote.audio (可选，说话者分离功能)
- Requires HF_TOKEN environment variable
- User must accept model user agreements at https://huggingface.co/pyannote/speaker-diarization-3.1

Next Phase Readiness

Phase 3 (ASR processing) is complete. Ready for Phase 4 (Analysis & Orchestration). Blockers/Concerns: None

7.1 KiB Raw Blame History