314 lines
9.9 KiB
Markdown
314 lines
9.9 KiB
Markdown
---
|
||
phase: 12-srt-simplifier
|
||
plan: 01
|
||
type: execute
|
||
wave: 1
|
||
depends_on: []
|
||
files_modified:
|
||
- kb_workflow/pipeline/processors/srt.py
|
||
- kb_workflow/pipeline/schemas.py
|
||
- kb_workflow/srt/__init__.py
|
||
- kb_workflow/srt/parser.py
|
||
- kb_workflow/srt/segmenter.py
|
||
- kb_workflow/srt/corrector.py
|
||
- tests/test_srt_processor.py
|
||
autonomous: true
|
||
requirements:
|
||
- SRT-01
|
||
- SRT-02
|
||
- SRT-03
|
||
- SRT-04
|
||
- OUT-01
|
||
- OUT-02
|
||
- OUT-04
|
||
must_haves:
|
||
truths:
|
||
- "SRT 文件能被正确解析为结构化数据"
|
||
- "字幕时间戳被移除，只保留纯文本"
|
||
- "文本按语义主题自动分段"
|
||
- "明显的语音识别错误被 LLM 纠正"
|
||
- "生成结构化的 Markdown 文件"
|
||
artifacts:
|
||
- path: "kb_workflow/srt/parser.py"
|
||
provides: "SRT 格式解析"
|
||
contains: "def parse_srt"
|
||
- path: "kb_workflow/srt/segmenter.py"
|
||
provides: "语义分段"
|
||
contains: "class SemanticSegmenter"
|
||
- path: "kb_workflow/srt/corrector.py"
|
||
provides: "LLM 纠错"
|
||
contains: "class ASRCorrector"
|
||
- path: "kb_workflow/pipeline/processors/srt.py"
|
||
provides: "Pipeline 处理器"
|
||
exports: ["SRTProcessor"]
|
||
key_links:
|
||
- from: "kb_workflow/pipeline/processors/srt.py"
|
||
to: "kb_workflow/srt/parser.py"
|
||
via: "parse_srt function"
|
||
pattern: "parse_srt"
|
||
- from: "kb_workflow/pipeline/processors/srt.py"
|
||
to: "kb_workflow/srt/segmenter.py"
|
||
via: "SemanticSegmenter class"
|
||
pattern: "SemanticSegmenter"
|
||
- from: "kb_workflow/pipeline/processors/srt.py"
|
||
to: "kb_workflow/srt/corrector.py"
|
||
via: "ASRCorrector class"
|
||
pattern: "ASRCorrector"
|
||
---
|
||
|
||
<objective>
|
||
创建 SRT 字幕处理模块，实现字幕解析、去时间戳、语义分段和 LLM 纠错功能。
|
||
|
||
Purpose: 将 B 站字幕文件转换为易读的 Markdown 文档，保留核心内容但移除时间戳等格式信息。
|
||
Output: SRT 处理模块（parser, segmenter, corrector）和 SRTProcessor 处理器。
|
||
</objective>
|
||
|
||
<execution_context>
|
||
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
|
||
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
|
||
</execution_context>
|
||
|
||
<context>
|
||
@.planning/PROJECT.md
|
||
@.planning/ROADMAP.md
|
||
@.planning/STATE.md
|
||
|
||
## 现有架构
|
||
|
||
### Pipeline 处理器模式（from base.py）
|
||
```python
|
||
class BaseProcessor(ABC):
|
||
STEP_NAME: str
|
||
|
||
def __init__(self, state_manager: Optional[WorkflowStateManager] = None):
|
||
self.settings = get_settings()
|
||
self.state_manager = state_manager
|
||
|
||
@abstractmethod
|
||
async def process(self, input_data: StepInput) -> StepOutput:
|
||
pass
|
||
```
|
||
|
||
### ASRProcessor 输出格式（from asr.py）
|
||
```python
|
||
class ASROutput(StepOutput):
|
||
tool: Literal["subtitle", "whisper", "whisper-diarization"]
|
||
has_subtitle: bool
|
||
output_files: Dict[str, str] # {txt, srt, json}
|
||
segments: List[Dict[str, Any]] # [{id, start, end, text}]
|
||
text: str # 完整文本
|
||
duration_seconds: float
|
||
speakers: Optional[List[str]]
|
||
```
|
||
|
||
### SRT 文件格式示例
|
||
```
|
||
1
|
||
00:00:00,000 --> 00:00:05,000
|
||
第一句字幕内容
|
||
|
||
2
|
||
00:00:05,000 --> 00:00:10,000
|
||
第二句字幕内容
|
||
```
|
||
|
||
### DeepAnalyzer AI 客户端模式（from deep_analyzer/__init__.py）
|
||
```python
|
||
class SimpleAIClient:
|
||
def __init__(self, api_key: str, provider: str, model: str, base_url: str):
|
||
self.api_key = api_key or os.environ.get("AI_API_KEY", "")
|
||
self.provider = provider or os.environ.get("AI_PROVIDER", "")
|
||
self.model = model or os.environ.get("AI_MODEL", "glm-5")
|
||
self.base_url = base_url or os.environ.get("AI_BASE_URL", "")
|
||
|
||
def generate(self, prompt: str, max_tokens: int = 16000) -> str:
|
||
# 支持 openai, zhipu, minimax 提供商
|
||
```
|
||
</context>
|
||
|
||
<tasks>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 1: 创建 SRT 解析器模块</name>
|
||
<files>kb_workflow/srt/__init__.py, kb_workflow/srt/parser.py, tests/test_srt_processor.py</files>
|
||
<behavior>
|
||
- Test 1: 标准的 SRT 格式能被正确解析为 Segment 列表
|
||
- Test 2: 包含特殊字符的字幕能正确处理
|
||
- Test 3: 空文件或格式错误的文件抛出 ValueError
|
||
</behavior>
|
||
<action>
|
||
创建 SRT 解析模块:
|
||
|
||
1. 创建 `kb_workflow/srt/__init__.py` 导出公共接口
|
||
2. 创建 `kb_workflow/srt/parser.py`:
|
||
- `@dataclass class Segment`: id, start_time, end_time, text
|
||
- `def parse_srt(content: str) -> List[Segment]`: 解析 SRT 格式
|
||
- `def parse_srt_file(file_path: Path) -> List[Segment]`: 从文件解析
|
||
- 使用正则表达式匹配 SRT 块: `(\d+)\n(\d{2}:\d{2}:\d{2},\d{3}) --> (\d{2}:\d{2}:\d{2},\d{3})\n(.*?)(?=\n\n|\Z)`
|
||
3. 创建单元测试文件 `tests/test_srt_processor.py`，测试解析功能
|
||
|
||
注意：时间戳格式为 `HH:MM:SS,mmm`，需要正确解析。
|
||
</action>
|
||
<verify>
|
||
<automated>pytest tests/test_srt_processor.py::test_parse_srt -v</automated>
|
||
</verify>
|
||
<done>
|
||
- SRT 文件能被解析为 List[Segment]
|
||
- 每个 Segment 包含 id, start_time, end_time, text
|
||
- 单元测试通过
|
||
</done>
|
||
</task>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 2: 创建语义分段器</name>
|
||
<files>kb_workflow/srt/segmenter.py, tests/test_srt_processor.py</files>
|
||
<behavior>
|
||
- Test 1: 连续短句被合并为段落
|
||
- Test 2: 主题变化时自动分段
|
||
- Test 3: 单个段落不超过 500 字符
|
||
</behavior>
|
||
<action>
|
||
创建语义分段模块:
|
||
|
||
1. 创建 `kb_workflow/srt/segmenter.py`:
|
||
- `class SemanticSegmenter`: 语义分段器
|
||
- `def segment(self, segments: List[Segment], max_paragraph_chars: int = 500) -> List[str]`:
|
||
将字幕分段转换为语义段落
|
||
- 分段策略:
|
||
a. 基于简单启发式规则（可选未来用 LLM 增强）
|
||
b. 空行/长停顿（>3秒）作为分段点
|
||
c. 段落长度限制（max_paragraph_chars）
|
||
- 返回段落文本列表（已去除时间戳）
|
||
|
||
2. 更新测试文件添加分段测试
|
||
|
||
不使用 LLM 进行分段判断（简化实现），仅基于时间和长度规则。
|
||
</action>
|
||
<verify>
|
||
<automated>pytest tests/test_srt_processor.py::test_segment -v</automated>
|
||
</verify>
|
||
<done>
|
||
- 字幕分段被合并为语义段落
|
||
- 时间戳被移除
|
||
- 段落长度合理（不超过 max_paragraph_chars）
|
||
</done>
|
||
</task>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 3: 创建 ASR 纠错器</name>
|
||
<files>kb_workflow/srt/corrector.py, tests/test_srt_processor.py</files>
|
||
<behavior>
|
||
- Test 1: 明显的同音字错误被纠正
|
||
- Test 2: 上下文相关的错误被纠正
|
||
- Test 3: 正确文本不被过度修改
|
||
</behavior>
|
||
<action>
|
||
创建 ASR 纠错模块:
|
||
|
||
1. 创建 `kb_workflow/srt/corrector.py`:
|
||
- `class ASRCorrector`: ASR 纠错器
|
||
- 复用 `SimpleAIClient`（from deep_analyzer）
|
||
- `def correct(self, text: str) -> str`: 纠正文本中的语音识别错误
|
||
- Prompt 设计:
|
||
```
|
||
你是一个文本校对专家。以下是语音识别生成的字幕，可能包含识别错误。
|
||
请只纠正明显的错误（如同音字、漏字、错字），不要改变原意和风格。
|
||
|
||
原文：
|
||
{text}
|
||
|
||
纠正后的文本（只输出纠正后的文本，不要解释）：
|
||
```
|
||
- 分批处理长文本（每批 5000 字符）
|
||
|
||
2. 更新测试文件添加纠错测试
|
||
|
||
注意：纠错是可选功能，如果 AI 不可用则返回原文。
|
||
</action>
|
||
<verify>
|
||
<automated>pytest tests/test_srt_processor.py::test_correct -v</automated>
|
||
</verify>
|
||
<done>
|
||
- ASR 纠错器能纠正明显的语音识别错误
|
||
- AI 不可用时返回原文
|
||
- 长文本被分批处理
|
||
</done>
|
||
</task>
|
||
|
||
<task type="auto">
|
||
<name>Task 4: 创建 SRTProcessor 和 Markdown 生成</name>
|
||
<files>kb_workflow/pipeline/processors/srt.py, kb_workflow/pipeline/schemas.py, kb_workflow/srt/__init__.py</files>
|
||
<action>
|
||
创建 SRTProcessor 处理器和输出 Schema:
|
||
|
||
1. 更新 `kb_workflow/pipeline/schemas.py`:
|
||
- `class SRTInput(StepInput)`: srt_file, metadata_file, output_file, content_type
|
||
- `class SRTOutput(StepOutput)`: output_file, paragraphs, tags, corrected
|
||
|
||
2. 创建 `kb_workflow/pipeline/processors/srt.py`:
|
||
- `class SRTProcessor(BaseProcessor)`: SRT 处理器
|
||
- `STEP_NAME = "srt"`
|
||
- `async def process(self, input_data: SRTInput) -> SRTOutput`:
|
||
a. 解析 SRT 文件（调用 parser.parse_srt_file）
|
||
b. 语义分段（调用 segmenter.segment）
|
||
c. ASR 纠错（调用 corrector.correct，可选）
|
||
d. 生成 Markdown 文件
|
||
|
||
3. Markdown 输出格式:
|
||
```markdown
|
||
---
|
||
title: "视频标题"
|
||
date: 2026-03-19
|
||
author: 作者名
|
||
platform: bilibili
|
||
duration: 12:34
|
||
bvid: BV1xx...
|
||
url: https://...
|
||
tags: ["标签1", "标签2"]
|
||
---
|
||
|
||
## 摘要
|
||
[LLM 生成的简短摘要，可选]
|
||
|
||
## 内容
|
||
段落1...
|
||
|
||
段落2...
|
||
|
||
...
|
||
```
|
||
|
||
4. 更新 `kb_workflow/srt/__init__.py` 导出所有公共接口
|
||
</action>
|
||
<verify>
|
||
<automated>pytest tests/test_srt_processor.py::test_srt_processor -v</automated>
|
||
</verify>
|
||
<done>
|
||
- SRTProcessor 能处理 SRT 文件生成 Markdown
|
||
- 输出包含 YAML frontmatter（元数据和标签）
|
||
- 处理器遵循 BaseProcessor 模式
|
||
</done>
|
||
</task>
|
||
|
||
</tasks>
|
||
|
||
<verification>
|
||
- 所有单元测试通过
|
||
- SRT 解析器能处理标准格式
|
||
- 语义分段器输出合理段落
|
||
- ASR 纠错器能调用 LLM
|
||
- SRTProcessor 生成有效 Markdown
|
||
</verification>
|
||
|
||
<success_criteria>
|
||
1. `parse_srt` 能将 SRT 内容解析为 Segment 列表
|
||
2. `SemanticSegmenter.segment` 能将字幕转换为语义段落
|
||
3. `ASRCorrector.correct` 能纠正语音识别错误
|
||
4. `SRTProcessor` 能生成包含元数据的 Markdown 文件
|
||
5. 所有测试通过（pytest tests/test_srt_processor.py）
|
||
</success_criteria>
|
||
|
||
<output>
|
||
After completion, create `.planning/phases/12-srt-simplifier/12-01-SUMMARY.md`
|
||
</output>
|