b312612e6c
- Create 12-01-SUMMARY.md with implementation details - Update STATE.md with current position (Phase 12, 50% complete) - Update ROADMAP.md with plan progress - Mark requirements SRT-01~04, OUT-01~02, OUT-04 as complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
180 lines
5.4 KiB
Markdown
180 lines
5.4 KiB
Markdown
---
|
|
phase: 12-srt-simplifier
|
|
plan: 01
|
|
subsystem: srt
|
|
tags: [srt, subtitle, parser, segmenter, corrector, pipeline]
|
|
dependencies:
|
|
requires: []
|
|
provides: [SRTProcessor, SemanticSegmenter, ASRCorrector, parse_srt]
|
|
affects: [kb_workflow.srt, kb_workflow.pipeline.processors]
|
|
tech_stack:
|
|
added: [dataclasses, asyncio, pydantic]
|
|
patterns: [TDD, BaseProcessor, Pipeline]
|
|
key_files:
|
|
created:
|
|
- kb_workflow/srt/__init__.py
|
|
- kb_workflow/srt/parser.py
|
|
- kb_workflow/srt/segmenter.py
|
|
- kb_workflow/srt/corrector.py
|
|
- kb_workflow/pipeline/processors/srt.py
|
|
- tests/test_srt_processor.py
|
|
modified:
|
|
- kb_workflow/pipeline/schemas.py
|
|
decisions:
|
|
- TDD approach with RED-GREEN cycles for parser, segmenter, corrector
|
|
- Lazy import for ASRCorrector to avoid dependency issues
|
|
- Simple rule-based segmentation (no LLM) for speed
|
|
- ASR correction is optional, fallback to original text
|
|
metrics:
|
|
duration: ~15 minutes
|
|
tasks_completed: 4
|
|
files_created: 6
|
|
files_modified: 1
|
|
tests_added: 20
|
|
test_pass_rate: 100%
|
|
---
|
|
|
|
# Phase 12 Plan 01: SRT Processing Module Summary
|
|
|
|
## One-liner
|
|
|
|
创建 SRT 字幕处理模块,实现字幕解析、语义分段和 LLM 纠错功能,将 B站字幕转换为易读的 Markdown 文档。
|
|
|
|
## Implementation Details
|
|
|
|
### Task 1: SRT Parser Module
|
|
|
|
**Files:** `kb_workflow/srt/__init__.py`, `kb_workflow/srt/parser.py`, `tests/test_srt_processor.py`
|
|
|
|
- Created `Segment` dataclass for subtitle segments
|
|
- Implemented `parse_srt()` function to parse SRT content string
|
|
- Implemented `parse_srt_file()` function to parse from file path
|
|
- Used regex pattern to match SRT blocks
|
|
- Handled special characters and multi-line subtitles
|
|
|
|
**Key Code:**
|
|
```python
|
|
@dataclass
|
|
class Segment:
|
|
id: int
|
|
start_time: float
|
|
end_time: float
|
|
text: str
|
|
|
|
def parse_srt(content: str) -> List[Segment]:
|
|
# Parse SRT format using regex
|
|
# Convert timestamps to seconds
|
|
# Return list of Segment objects
|
|
```
|
|
|
|
### Task 2: Semantic Segmenter
|
|
|
|
**Files:** `kb_workflow/srt/segmenter.py`, `tests/test_srt_processor.py`
|
|
|
|
- Created `SemanticSegmenter` class for paragraph segmentation
|
|
- Implemented time-gap based splitting (>3 seconds)
|
|
- Added max paragraph length constraint (500 chars)
|
|
- Simple rule-based approach (no LLM needed)
|
|
|
|
**Key Code:**
|
|
```python
|
|
class SemanticSegmenter:
|
|
def __init__(self, gap_threshold=3.0, max_paragraph_chars=500):
|
|
...
|
|
|
|
def segment(self, segments: List[Segment]) -> List[str]:
|
|
# Split on time gaps
|
|
# Enforce paragraph length limits
|
|
# Return list of paragraph strings
|
|
```
|
|
|
|
### Task 3: ASR Corrector
|
|
|
|
**Files:** `kb_workflow/srt/corrector.py`, `tests/test_srt_processor.py`
|
|
|
|
- Created `ASRCorrector` class for speech recognition error correction
|
|
- Reuses `SimpleAIClient` from deep_analyzer
|
|
- Implemented batch processing for long text (5000 chars per batch)
|
|
- Fallback to original text when AI unavailable
|
|
|
|
**Key Code:**
|
|
```python
|
|
class ASRCorrector:
|
|
def __init__(self, ai_client=None, batch_size=5000):
|
|
...
|
|
|
|
def correct(self, text: str) -> str:
|
|
# Use LLM to correct obvious errors
|
|
# Batch process long text
|
|
# Return corrected text or original
|
|
```
|
|
|
|
### Task 4: SRTProcessor and Markdown Generation
|
|
|
|
**Files:** `kb_workflow/pipeline/processors/srt.py`, `kb_workflow/pipeline/schemas.py`, `tests/test_srt_processor.py`
|
|
|
|
- Created `SRTProcessor` following BaseProcessor pattern
|
|
- Added `SRTInput` and `SRTOutput` schemas
|
|
- Generated Markdown with YAML frontmatter
|
|
- Handled missing metadata gracefully
|
|
|
|
**Key Code:**
|
|
```python
|
|
class SRTProcessor(BaseProcessor):
|
|
STEP_NAME = "srt"
|
|
|
|
async def process(self, input_data: SRTInput) -> SRTOutput:
|
|
# 1. Parse SRT file
|
|
# 2. Semantic segmentation
|
|
# 3. ASR correction (optional)
|
|
# 4. Generate Markdown
|
|
```
|
|
|
|
## Deviations from Plan
|
|
|
|
None - plan executed exactly as written.
|
|
|
|
## Test Coverage
|
|
|
|
| Module | Tests | Coverage |
|
|
|--------|-------|----------|
|
|
| Parser | 6 | parse_srt, parse_srt_file, error handling |
|
|
| Segmenter | 4 | merge, split on gap, max length, empty |
|
|
| Corrector | 6 | correction, fallback, batch, empty |
|
|
| Processor | 4 | markdown generation, frontmatter, pattern, missing metadata |
|
|
| **Total** | **20** | **100% pass rate** |
|
|
|
|
## Files Created/Modified
|
|
|
|
### Created
|
|
- `kb_workflow/srt/__init__.py` - Module exports
|
|
- `kb_workflow/srt/parser.py` - SRT parsing logic
|
|
- `kb_workflow/srt/segmenter.py` - Semantic segmentation
|
|
- `kb_workflow/srt/corrector.py` - ASR error correction
|
|
- `kb_workflow/pipeline/processors/srt.py` - Pipeline processor
|
|
- `tests/test_srt_processor.py` - Unit tests
|
|
|
|
### Modified
|
|
- `kb_workflow/pipeline/schemas.py` - Added SRTInput/SRTOutput schemas
|
|
|
|
## Commits
|
|
|
|
| Commit | Description |
|
|
|--------|-------------|
|
|
| 7023a57 | test(12-01): add failing tests for SRT parser module |
|
|
| c259e0a | feat(12-01): implement SRT parser module |
|
|
| 81a7dc5 | test(12-01): add failing tests for SemanticSegmenter |
|
|
| a350693 | feat(12-01): implement SemanticSegmenter for subtitle segmentation |
|
|
| 757793b | test(12-01): add failing tests for ASRCorrector |
|
|
| 5e00632 | feat(12-01): implement ASRCorrector for speech recognition error correction |
|
|
| 87b1919 | feat(12-01): implement SRTProcessor and Markdown generation |
|
|
| 812f016 | chore(12-01): update srt module exports |
|
|
|
|
## Next Steps
|
|
|
|
Phase 12 Plan 02 will:
|
|
- Integrate SRTProcessor into Pipeline
|
|
- Add platform routing for B站视频
|
|
- Replace DeepAnalyzer for B站 videos
|
|
- End-to-end testing
|