382 lines
13 KiB
Markdown
382 lines
13 KiB
Markdown
---
|
||
phase: 14-robustness-cost
|
||
plan: 05
|
||
type: execute
|
||
wave: 2
|
||
depends_on:
|
||
- 14-01
|
||
- 14-02
|
||
- 14-03
|
||
- 14-04
|
||
files_modified:
|
||
- kb_workflow/srt/semantic_engine.py
|
||
- kb_workflow/srt/processor.py
|
||
autonomous: true
|
||
requirements:
|
||
- COST-03
|
||
must_haves:
|
||
truths:
|
||
- "用户可通过配置开关启用/禁用 LLM 分段"
|
||
- "禁用时使用简单规则分段，不影响其他平台"
|
||
- "SRTProcessor.enable_semantic_segmentation 参数控制开关"
|
||
artifacts:
|
||
- path: "kb_workflow/srt/processor.py"
|
||
provides: "SRTProcessor 配置开关"
|
||
exports: ["SRTProcessor.__init__"]
|
||
key_links:
|
||
- from: "SRTProcessor.__init__()"
|
||
to: "SemanticSegmentationEngine"
|
||
via: "enable_semantic_segmentation 参数"
|
||
pattern: "if enable_semantic_segmentation"
|
||
---
|
||
|
||
<objective>
|
||
集成所有鲁棒性和成本控制功能到 SRTProcessor，提供配置开关控制 LLM 分段。
|
||
|
||
Purpose: 完成阶段 14 的集成，确保所有功能可通过配置开关控制。
|
||
Output: 集成缓存、分块、重试、降级的 SRTProcessor，支持配置开关。
|
||
</objective>
|
||
|
||
<execution_context>
|
||
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
|
||
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
|
||
</execution_context>
|
||
|
||
<context>
|
||
@.planning/phases/14-robustness-cost/14-CONTEXT.md
|
||
@.planning/phases/14-robustness-cost/14-RESEARCH.md
|
||
@.planning/phases/14-robustness-cost/14-VALIDATION.md
|
||
@.planning/ROADMAP.md
|
||
@.planning/REQUIREMENTS.md
|
||
@.planning/STATE.md
|
||
|
||
@kb_workflow/srt/processor.py
|
||
@kb_workflow/srt/semantic_engine.py
|
||
@.planning/phases/14-robustness-cost/14-01-PLAN.md
|
||
@.planning/phases/14-robustness-cost/14-02-PLAN.md
|
||
@.planning/phases/14-robustness-cost/14-03-PLAN.md
|
||
@.planning/phases/14-robustness-cost/14-04-PLAN.md
|
||
</context>
|
||
|
||
<interfaces>
|
||
<!-- Key contracts from processor.py for integration -->
|
||
|
||
From kb_workflow/srt/processor.py (Phase 13 Plan 07):
|
||
```python
|
||
class SRTProcessor(BaseProcessor):
|
||
def __init__(
|
||
self,
|
||
config: Optional[dict] = None,
|
||
enable_semantic_segmentation: bool = False,
|
||
**kwargs
|
||
):
|
||
"""初始化 SRT 处理器
|
||
|
||
Args:
|
||
config: 配置字典
|
||
enable_semantic_segmentation: 是否启用语义分段（默认 False）
|
||
"""
|
||
super().__init__(config=config, **kwargs)
|
||
|
||
# 初始化子模块
|
||
self.parser = SRTParser()
|
||
self.corrector = ASRCorrector()
|
||
|
||
# 语义分段开关
|
||
self.enable_semantic_segmentation = enable_semantic_segmentation
|
||
if self.enable_semantic_segmentation:
|
||
self.semantic_engine = SemanticSegmentationEngine()
|
||
else:
|
||
self.semantic_engine = None
|
||
self.segmenter = SemanticSegmenter()
|
||
|
||
def _segment_subtitles(self, segments: List, metadata: dict) -> List[str]:
|
||
"""分段字幕
|
||
|
||
Args:
|
||
segments: 字幕分段列表
|
||
metadata: 视频元数据
|
||
|
||
Returns:
|
||
分段后的文本列表
|
||
"""
|
||
if self.enable_semantic_segmentation and self.semantic_engine:
|
||
# 使用 LLM 语义分段
|
||
video_segments = self.semantic_engine.segment(segments, metadata)
|
||
# 转换为纯文本列表（保留标题）
|
||
paragraphs = []
|
||
for seg in video_segments:
|
||
if seg.title:
|
||
paragraphs.append(f"## {seg.title}\n\n{seg.content}")
|
||
else:
|
||
paragraphs.append(seg.content)
|
||
return paragraphs
|
||
else:
|
||
# 使用规则分段
|
||
return self.segmenter.segment(segments)
|
||
```
|
||
|
||
关键观察:
|
||
- enable_semantic_segmentation 参数控制是否启用 LLM 分段
|
||
- True: 使用 SemanticSegmentationEngine
|
||
- False: 使用 SemanticSegmenter (规则分段)
|
||
- 输出格式: 带标题的分段使用 "## {title}\n\n{content}" 格式
|
||
|
||
From 14-CONTEXT.md (User Decisions):
|
||
```python
|
||
# Claude's Discretion
|
||
- 具体的重试逻辑实现细节 (Plan 01)
|
||
- 缓存文件命名规则和目录结构 (Plan 03)
|
||
- 分块算法的具体实现 (Plan 04)
|
||
- 日志消息的具体措辞和格式 (Plan 01, 02)
|
||
```
|
||
|
||
本计划需要:
|
||
1. 集成 SemanticCache 到 UnifiedSegmentationStrategy
|
||
2. 确保配置开关 (enable_semantic_segmentation) 正常工作
|
||
3. 添加集成测试验证完整流程
|
||
</interfaces>
|
||
|
||
<tasks>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 1: 集成缓存到 UnifiedSegmentationStrategy</name>
|
||
<files>kb_workflow/srt/semantic_engine.py</files>
|
||
<behavior>
|
||
- UnifiedSegmentationStrategy 接受可选的 cache 参数
|
||
- 在 segment() 方法中，调用 LLM 前先检查缓存
|
||
- 缓存命中: 直接返回缓存结果
|
||
- 缓存未命中: 调用 LLM，然后保存结果到缓存
|
||
- 缓存键仅基于字幕内容，不包含元数据
|
||
</behavior>
|
||
<action>
|
||
修改 UnifiedSegmentationStrategy 类，集成缓存:
|
||
|
||
1. 修改 __init__() 方法，添加 cache 参数:
|
||
```python
|
||
def __init__(
|
||
self,
|
||
ai_client: Optional[AIClientProtocol] = None,
|
||
fallback_segmenter: Optional['SemanticSegmenter'] = None,
|
||
cache: Optional['SemanticCache'] = None
|
||
):
|
||
"""初始化统一分段策略
|
||
|
||
Args:
|
||
ai_client: AI 客户端实例
|
||
fallback_segmenter: 降级时使用的规则分段器
|
||
cache: 缓存管理器，如果为 None 则不使用缓存
|
||
"""
|
||
self.ai_client = ai_client
|
||
self.fallback_segmenter = fallback_segmenter
|
||
self.cache = cache
|
||
|
||
# Lazy AI Client 初始化（保持不变）
|
||
if self.ai_client is None:
|
||
try:
|
||
from kb_workflow.deep_analyzer import SimpleAIClient
|
||
self.ai_client = SimpleAIClient()
|
||
if not self.ai_client.api_key:
|
||
logger.info("AI API key not configured")
|
||
self.ai_client = None
|
||
except Exception as e:
|
||
logger.warning(f"Failed to initialize AI client: {e}")
|
||
self.ai_client = None
|
||
|
||
# 初始化降级分段器
|
||
if self.fallback_segmenter is None:
|
||
from kb_workflow.srt.segmenter import SemanticSegmenter
|
||
self.fallback_segmenter = SemanticSegmenter()
|
||
```
|
||
|
||
2. 修改 segment() 方法，添加缓存逻辑:
|
||
```python
|
||
def segment(self, segments: List, metadata: dict) -> List[VideoSegment]:
|
||
# AI 不可用，直接使用规则分段
|
||
if self.ai_client is None:
|
||
logger.info("AI client unavailable, using fallback segmenter")
|
||
paragraphs = self.fallback_segmenter.segment(segments)
|
||
return [
|
||
VideoSegment(title=None, content=para, start_index=0, end_index=0)
|
||
for para in paragraphs
|
||
]
|
||
|
||
# 合并字幕文本
|
||
content = " ".join(seg.text for seg in segments)
|
||
|
||
# 检查缓存
|
||
if self.cache is not None:
|
||
cached_result = self.cache.get(content)
|
||
if cached_result is not None:
|
||
logger.info("Using cached segmentation result")
|
||
return cached_result.segments
|
||
|
||
# 检查是否需要分块
|
||
if len(content) <= 5000:
|
||
# 短文本直接处理
|
||
video_segments = self._segment_single_chunk(content, metadata, segments)
|
||
else:
|
||
# 长文本分块处理
|
||
video_segments = self._segment_multiple_chunks(content, metadata, segments)
|
||
|
||
# 保存到缓存
|
||
if self.cache is not None:
|
||
# 构建完整的 SegmentationResult
|
||
result = SegmentationResult(
|
||
video_type=self._infer_video_type(metadata),
|
||
segments=video_segments,
|
||
actual_segment_count=len(video_segments)
|
||
)
|
||
self.cache.set(content, result)
|
||
logger.debug("Saved segmentation result to cache")
|
||
|
||
return video_segments
|
||
```
|
||
|
||
3. 修改 SemanticSegmentationEngine.__init__()，传递 cache 参数:
|
||
```python
|
||
def __init__(
|
||
self,
|
||
strategy: Optional[SegmentationStrategy] = None,
|
||
fallback_segmenter: Optional['SemanticSegmenter'] = None,
|
||
cache: Optional['SemanticCache'] = None
|
||
):
|
||
"""初始化语义分段引擎
|
||
|
||
Args:
|
||
strategy: 分段策略实例
|
||
fallback_segmenter: 降级时使用的规则分段器（已弃用）
|
||
cache: 缓存管理器
|
||
"""
|
||
if strategy is None:
|
||
self.strategy = UnifiedSegmentationStrategy(
|
||
fallback_segmenter=fallback_segmenter,
|
||
cache=cache
|
||
)
|
||
else:
|
||
self.strategy = strategy
|
||
```
|
||
|
||
遵守用户决策:
|
||
- 缓存是可选的，通过构造函数传入
|
||
- 缓存键仅基于内容哈希
|
||
- 缓存命中时记录 info 日志
|
||
</action>
|
||
<verify>
|
||
<automated>pytest tests/test_semantic_robustness.py::test_cache_hit -x</automated>
|
||
</verify>
|
||
<done>
|
||
- UnifiedSegmentationStrategy 支持可选缓存
|
||
- 缓存命中时直接返回，不调用 LLM
|
||
- 缓存未命中时调用 LLM 并保存结果
|
||
- SemanticSegmentationEngine 支持传递 cache 参数
|
||
</done>
|
||
</task>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 2: 在 SRTProcessor 中集成缓存和验证配置开关</name>
|
||
<files>kb_workflow/srt/processor.py</files>
|
||
<behavior>
|
||
- SRTProcessor 接受可选的 enable_cache 参数
|
||
- 启用语义分段时创建 SemanticCache 实例
|
||
- 禁用语义分段时不创建缓存
|
||
- 配置开关正确控制 LLM 分段和规则分段
|
||
</behavior>
|
||
<action>
|
||
修改 SRTProcessor 类，集成缓存:
|
||
|
||
1. 修改 __init__() 方法，添加缓存支持:
|
||
```python
|
||
def __init__(
|
||
self,
|
||
config: Optional[dict] = None,
|
||
enable_semantic_segmentation: bool = False,
|
||
enable_cache: bool = True,
|
||
**kwargs
|
||
):
|
||
"""初始化 SRT 处理器
|
||
|
||
Args:
|
||
config: 配置字典
|
||
enable_semantic_segmentation: 是否启用语义分段（默认 False）
|
||
enable_cache: 是否启用缓存（默认 True，仅在启用语义分段时有效）
|
||
"""
|
||
super().__init__(config=config, **kwargs)
|
||
|
||
# 初始化子模块
|
||
self.parser = SRTParser()
|
||
self.corrector = ASRCorrector()
|
||
|
||
# 语义分段开关
|
||
self.enable_semantic_segmentation = enable_semantic_segmentation
|
||
if self.enable_semantic_segmentation:
|
||
# 初始化缓存
|
||
cache = None
|
||
if enable_cache:
|
||
from kb_workflow.srt.cache import SemanticCache
|
||
cache = SemanticCache()
|
||
|
||
self.semantic_engine = SemanticSegmentationEngine(cache=cache)
|
||
self.segmenter = None
|
||
else:
|
||
self.semantic_engine = None
|
||
self.segmenter = SemanticSegmenter()
|
||
|
||
logger.info("Using rule-based segmenter (semantic segmentation disabled)")
|
||
```
|
||
|
||
2. 验证 _segment_subtitles() 方法正确使用配置开关:
|
||
- 检查 enable_semantic_segmentation 为 True 时使用 SemanticSegmentationEngine
|
||
- 检查 enable_semantic_segmentation 为 False 时使用 SemanticSegmenter
|
||
- 输出格式正确（带标题的分段使用 "## {title}\n\n{content}"）
|
||
|
||
遵守用户决策:
|
||
- 配置开关控制是否启用 LLM 分段
|
||
- 默认禁用语义分段（向后兼容）
|
||
- 缓存默认启用（当语义分段启用时）
|
||
</action>
|
||
<verify>
|
||
<automated>pytest tests/test_srt_processor.py::test_semantic_segmentation_toggle -x</automated>
|
||
</verify>
|
||
<done>
|
||
- SRTProcessor 支持配置开关控制语义分段
|
||
- 启用时创建 SemanticCache 和 SemanticSegmentationEngine
|
||
- 禁用时使用 SemanticSegmenter，不创建缓存
|
||
- 集成测试验证完整流程
|
||
</done>
|
||
</task>
|
||
|
||
</tasks>
|
||
|
||
<verification>
|
||
## 整体验证标准
|
||
|
||
1. **配置开关**: enable_semantic_segmentation=True 启用 LLM 分段，False 使用规则分段
|
||
2. **缓存集成**: 启用语义分段时缓存工作，禁用时不创建缓存
|
||
3. **向后兼容**: 默认禁用语义分段，不影响现有功能
|
||
4. **完整流程**: 从 SRT 处理到 Markdown 生成的完整流程正常工作
|
||
|
||
运行完整测试套件:
|
||
```bash
|
||
python3 -m pytest tests/test_srt_processor.py::test_semantic_segmentation_toggle -v -x
|
||
```
|
||
|
||
运行鲁棒性测试:
|
||
```bash
|
||
python3 -m pytest tests/test_semantic_robustness.py -v -x
|
||
```
|
||
</verification>
|
||
|
||
<success_criteria>
|
||
1. enable_semantic_segmentation=True 时使用 LLM 分段
|
||
2. enable_semantic_segmentation=False 时使用规则分段
|
||
3. 缓存在启用语义分段时正常工作
|
||
4. 禁用语义分段时不创建缓存
|
||
5. 所有功能集成测试通过
|
||
6. 所有鲁棒性测试通过
|
||
</success_criteria>
|
||
|
||
<output>
|
||
执行完成后创建 `.planning/phases/14-robustness-cost/14-05-SUMMARY.md`
|
||
</output>
|