docs(13-07): complete SRTProcessor integration plan
- Create SUMMARY.md documenting SRTProcessor integration with LLM semantic segmentation - Update STATE.md with completed plan position and performance metrics - Update ROADMAP.md marking Phase 13 plans 01-07 as complete Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -166,13 +166,13 @@ Plans:
|
||||
**Plans**: 7 plans
|
||||
|
||||
Plans:
|
||||
- [ ] 13-01-PLAN.md — 视频类型识别（Pydantic 模型、类型推断、LazyAIClient 初始化）
|
||||
- [ ] 13-02-PLAN.md — 语义边界检测（Prompt 模板、JSON 提取、Pydantic 验证）
|
||||
- [ ] 13-03-PLAN.md — 内容完整性保护（句边界、论点完整性 Prompt 指令）
|
||||
- [ ] 13-04-PLAN.md — 主题标题生成（title 字段、验证、Prompt 指令）
|
||||
- [ ] 13-05-PLAN.md — 中文句边界检测（列举标点符号、语义停顿 Prompt 指令）
|
||||
- [ ] 13-06-PLAN.md — 可扩展策略架构（Strategy Pattern、统一策略实现）
|
||||
- [ ] 13-07-PLAN.md — SRTProcessor 集成（enable_semantic_segmentation 参数、降级策略）
|
||||
- [x] 13-01-PLAN.md — 视频类型识别（Pydantic 模型、类型推断、LazyAIClient 初始化）
|
||||
- [x] 13-02-PLAN.md — 语义边界检测（Prompt 模板、JSON 提取、Pydantic 验证）
|
||||
- [x] 13-03-PLAN.md — 内容完整性保护（句边界、论点完整性 Prompt 指令）
|
||||
- [x] 13-04-PLAN.md — 主题标题生成（title 字段、验证、Prompt 指令）
|
||||
- [x] 13-05-PLAN.md — 中文句边界检测（列举标点符号、语义停顿 Prompt 指令）
|
||||
- [x] 13-06-PLAN.md — 可扩展策略架构（Strategy Pattern、统一策略实现）
|
||||
- [x] 13-07-PLAN.md — SRTProcessor 集成（enable_semantic_segmentation 参数、降级策略）
|
||||
|
||||
### Phase 14: 鲁棒性与成本控制
|
||||
**Goal**: 提高系统稳定性，实现优雅降级和成本优化
|
||||
|
||||
+12
-15
@@ -3,15 +3,15 @@ gsd_state_version: 1.0
|
||||
milestone: v1.4
|
||||
milestone_name: Zhihu Anti-Crawler
|
||||
status: executing
|
||||
stopped_at: Completed Phase 13 Plan 06 - Extensible Strategy Architecture
|
||||
last_updated: "2026-03-21T16:30:40.235Z"
|
||||
last_activity: 2026-03-22 — Phase 13 Plan 06 completed (extensible strategy architecture)
|
||||
stopped_at: Completed Phase 13 Plan 07 - SRTProcessor Integration
|
||||
last_updated: "2026-03-21T16:32:11.000Z"
|
||||
last_activity: 2026-03-22 — Phase 13 Plan 07 completed (SRTProcessor integration)
|
||||
progress:
|
||||
total_phases: 15
|
||||
completed_phases: 10
|
||||
total_plans: 38
|
||||
completed_plans: 36
|
||||
percent: 30
|
||||
completed_plans: 37
|
||||
percent: 31
|
||||
---
|
||||
|
||||
# Project State
|
||||
@@ -26,11 +26,11 @@ See: .planning/PROJECT.md (updated 2026-03-21)
|
||||
## Current Position
|
||||
|
||||
Phase: Phase 13 - 核心 LLM 分段功能
|
||||
Plan: 07
|
||||
Plan: 07 (completed)
|
||||
Status: In progress
|
||||
Last activity: 2026-03-22 — Phase 13 Plan 06 completed (extensible strategy architecture)
|
||||
Last activity: 2026-03-22 — Phase 13 Plan 07 completed (SRTProcessor integration)
|
||||
|
||||
Progress: [██░░░░░░░░░] 30% (Phase 13/15, 6/7 plans complete)
|
||||
Progress: [██░░░░░░░░░] 31% (Phase 13/15, 7/7 plans complete)
|
||||
|
||||
### Phase 13 Goal
|
||||
实现基于 LLM 的智能语义分段，识别视频类型并生成主题标题
|
||||
@@ -131,18 +131,15 @@ None.
|
||||
| 13 | 04 | 2min | 1 | 2 | 2026-03-22 |
|
||||
| 13 | 05 | 2min | 1 | 2 | 2026-03-22 |
|
||||
| 13 | 06 | 12min | 1 | 2 | 2026-03-22 |
|
||||
| Phase 13 P01 | 35min | 1 tasks | 4 files |
|
||||
| Phase 13 P02 | 4min | 1 tasks | 2 files |
|
||||
| Phase 13 P06 | 12min | 1 tasks | 2 files |
|
||||
| Phase 13 P06 | 1774110638min | 1 tasks | 2 files |
|
||||
| 13 | 07 | 4min | 1 | 4 | 2026-03-22 |
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-03-21T16:30:40.233Z
|
||||
Stopped at: Completed Phase 13 Plan 06 - Extensible Strategy Architecture
|
||||
Last session: 2026-03-21T16:32:11.000Z
|
||||
Stopped at: Completed Phase 13 Plan 07 - SRTProcessor Integration
|
||||
Resume file: None
|
||||
|
||||
**Next step:** Continue with Phase 13 remaining plans (03-07)
|
||||
**Next step:** Phase 13 complete, continue to Phase 14 (鲁棒性) or Phase 15 (优化)
|
||||
|
||||
## Phase 13 Plan Structure
|
||||
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
---
|
||||
phase: 13-llm
|
||||
plan: 07
|
||||
subsystem: srt-processor-integration
|
||||
tags: [semantic-segmentation, llm-integration, srt-processor, video-segment, fallback-mechanism]
|
||||
|
||||
# Dependency graph
|
||||
requires:
|
||||
- phase: 13-01
|
||||
provides: video-type-detection
|
||||
- phase: 13-02
|
||||
provides: semantic-boundary-detection
|
||||
- phase: 13-03
|
||||
provides: content-completeness-protection
|
||||
- phase: 13-04
|
||||
provides: title-generation
|
||||
- phase: 13-05
|
||||
provides: chinese-sentence-boundary
|
||||
- phase: 13-06
|
||||
provides: strategy-pattern-architecture
|
||||
provides:
|
||||
- SRTProcessor 集成 LLM 语义分段（enable_semantic_segmentation 参数）
|
||||
- 支持带标题的分段输出（VideoSegment → Markdown H2）
|
||||
- AI 不可用时自动降级到规则分段
|
||||
- 向后兼容（enable_semantic_segmentation=False 时行为不变）
|
||||
- 全面的集成测试覆盖（9 个测试用例）
|
||||
affects: [phase-14-robustness, phase-15-optimization]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: [strategy-integration, lazy-initialization, graceful-degradation, backward-compatibility]
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- tests/test_srt_processor_integration.py (345 lines)
|
||||
modified:
|
||||
- kb_workflow/pipeline/processors/srt.py (393 lines)
|
||||
- kb_workflow/srt/semantic_engine.py (587 lines)
|
||||
- tests/test_semantic_engine.py (992 lines)
|
||||
|
||||
key-decisions:
|
||||
- "UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment] 而非 List[str] 以保留标题信息"
|
||||
- "调整 SRTProcessor 步骤顺序：先加载元数据，再进行语义分段（LLM 需要 metadata）"
|
||||
- "在 SemanticSegmentationEngine 中添加代理方法以保持向后兼容性"
|
||||
|
||||
patterns-established:
|
||||
- "Strategy Integration: SRTProcessor 通过 SemanticSegmentationEngine 使用 UnifiedSegmentationStrategy"
|
||||
- "Graceful Degradation: AI 不可用时自动降级到规则分段，记录日志"
|
||||
- "Backward Compatibility: enable_semantic_segmentation=False 时保持原有行为"
|
||||
- "Type Flexibility: _generate_markdown() 同时支持 List[str] 和 List[VideoSegment]"
|
||||
|
||||
requirements-completed: [CONFIG-03]
|
||||
|
||||
# Metrics
|
||||
duration: 4min
|
||||
started: 2026-03-21T16:28:25Z
|
||||
completed: 2026-03-21T16:32:11Z
|
||||
tasks: 1
|
||||
files: 4
|
||||
---
|
||||
|
||||
# Phase 13 Plan 07: SRTProcessor Integration Summary
|
||||
|
||||
**SRTProcessor 集成 LLM 语义分段，支持配置开关控制、自动降级和带标题的 Markdown 输出**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 4 min
|
||||
- **Started:** 2026-03-21T16:28:25Z
|
||||
- **Completed:** 2026-03-21T16:32:11Z
|
||||
- **Tasks:** 1
|
||||
- **Files modified:** 4
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- SRTProcessor 支持 `enable_semantic_segmentation` 参数控制分段模式
|
||||
- 启用时使用 SemanticSegmentationEngine 进行 LLM 语义分段，禁用时使用规则分段
|
||||
- AI 不可用时自动降级到规则分段，记录日志
|
||||
- `_generate_markdown()` 更新为支持带标题的 VideoSegment 输出（H2 标题）
|
||||
- 调整步骤顺序：先加载元数据，再进行语义分段（LLM 需要 metadata 构建 Prompt）
|
||||
- UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment] 以保留标题信息
|
||||
- 添加代理方法到 SemanticSegmentationEngine（_infer_video_type, _build_prompt）以保持向后兼容
|
||||
- 创建全面的集成测试（9 个测试用例）：语义分段模式、规则分段模式、降级场景、输出格式验证
|
||||
- 更新现有语义引擎测试以适配新的 VideoSegment 返回类型
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: 集成 SemanticSegmentationEngine 到 SRTProcessor** - `ef4a660` (feat)
|
||||
|
||||
**Plan metadata:** (pending final commit)
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `kb_workflow/pipeline/processors/srt.py` - 添加 enable_semantic_segmentation 参数，集成 SemanticSegmentationEngine，更新 _generate_markdown() 支持带标题的输出，调整步骤顺序
|
||||
- `kb_workflow/srt/semantic_engine.py` - 更新 UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment]，添加代理方法到 SemanticSegmentationEngine
|
||||
- `tests/test_srt_processor_integration.py` - 新增集成测试文件，包含 9 个测试用例覆盖所有场景
|
||||
- `tests/test_semantic_engine.py` - 更新现有测试以适配新的 VideoSegment 返回类型和策略模式 API
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 1 - Bug] 修改 UnifiedSegmentationStrategy.segment() 返回类型**
|
||||
- **Found during:** Task 1 (集成 SRTProcessor)
|
||||
- **Issue:** 计划要求 SRTProcessor 支持带标题的分段输出，但 UnifiedSegmentationStrategy.segment() 返回 List[str]，标题信息丢失
|
||||
- **Fix:** 修改 UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment]，在降级时将规则分段结果转换为 VideoSegment 格式（title=None）
|
||||
- **Files modified:** kb_workflow/srt/semantic_engine.py
|
||||
- **Verification:** 集成测试验证带标题的输出正确生成
|
||||
- **Committed in:** ef4a660 (Task 1 commit)
|
||||
|
||||
**2. [Rule 3 - Blocking] 调整 SRTProcessor 步骤顺序**
|
||||
- **Found during:** Task 1 (实现语义分段)
|
||||
- **Issue:** 代码在分段时使用 metadata 变量，但 metadata 在分段后才加载（UnboundLocalError）
|
||||
- **Fix:** 调整步骤顺序：先加载元数据，再进行语义分段，确保 LLM 可以使用 metadata 构建 Prompt
|
||||
- **Files modified:** kb_workflow/pipeline/processors/srt.py
|
||||
- **Verification:** 测试验证语义分段正确使用元数据
|
||||
- **Committed in:** ef4a660 (Task 1 commit)
|
||||
|
||||
**3. [Rule 3 - Blocking] 修复 SRTOutput Schema 验证错误**
|
||||
- **Found during:** Task 1 (返回处理结果)
|
||||
- **Issue:** SRTOutput 期望 paragraphs 是 List[str]，但语义分段返回 List[VideoSegment]（Pydantic ValidationError）
|
||||
- **Fix:** 在返回 SRTOutput 之前将 VideoSegment 转换为纯文本（提取 .content 字段）
|
||||
- **Files modified:** kb_workflow/pipeline/processors/srt.py
|
||||
- **Verification:** 集成测试验证 SRTOutput 正确返回
|
||||
- **Committed in:** ef4a660 (Task 1 commit)
|
||||
|
||||
**4. [Rule 1 - Bug] 更新 ASR 纠错以处理 VideoSegment**
|
||||
- **Found during:** Task 1 (实现 ASR 纠错步骤)
|
||||
- **Issue:** ASR 纠错代码期望纯文本，但语义分段返回 VideoSegment 对象
|
||||
- **Fix:** 更新 ASR 纠错代码，检查段落类型（VideoSegment 或纯文本），相应地提取和更新内容
|
||||
- **Files modified:** kb_workflow/pipeline/processors/srt.py
|
||||
- **Verification:** 测试验证 ASR 纠错正确处理两种类型
|
||||
- **Committed in:** ef4a660 (Task 1 commit)
|
||||
|
||||
**5. [Rule 1 - Bug] 添加代理方法到 SemanticSegmentationEngine**
|
||||
- **Found during:** Task 1 (运行现有测试)
|
||||
- **Issue:** 现有测试期望 SemanticSegmentationEngine 有 _infer_video_type 和 _build_prompt 方法，但策略模式重构后这些方法移到了 UnifiedSegmentationStrategy
|
||||
- **Fix:** 在 SemanticSegmentationEngine 中添加代理方法，委托给 self.strategy
|
||||
- **Files modified:** kb_workflow/srt/semantic_engine.py
|
||||
- **Verification:** 现有语义引擎测试全部通过（39 个测试）
|
||||
- **Committed in:** ef4a660 (Task 1 commit)
|
||||
|
||||
**6. [Rule 1 - Bug] 更新测试以适配 VideoSegment 返回类型**
|
||||
- **Found during:** Task 1 (运行现有测试)
|
||||
- **Issue:** 测试代码期望 paragraphs 是字符串列表，但现在返回 VideoSegment 对象列表（AttributeError: 'VideoSegment' object has no attribute 'strip'）
|
||||
- **Fix:** 更新所有测试以访问 .content 属性（paragraphs[0].content 而非 paragraphs[0]）
|
||||
- **Files modified:** tests/test_semantic_engine.py
|
||||
- **Verification:** 所有语义引擎测试通过（39 个测试）
|
||||
- **Committed in:** ef4a660 (Task 1 commit)
|
||||
|
||||
---
|
||||
|
||||
**Total deviations:** 6 auto-fixed (4 bugs, 2 blocking)
|
||||
**Impact on plan:** 所有自动修复都是必要的，以确保正确性和功能完整性。没有范围蔓延。
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
**1. 策略模式 API 变更导致测试失败**
|
||||
- **问题:** Plan 13-06 引入策略模式后，SemanticSegmentationEngine.__init__() 不再接受 ai_client 参数，导致现有测试失败
|
||||
- **解决:** 更新所有测试使用新的策略模式 API（创建 UnifiedSegmentationStrategy，然后注入到 SemanticSegmentationEngine）
|
||||
- **影响:** 需要更新 tests/test_semantic_engine.py 中的多处代码
|
||||
|
||||
**2. 返回类型变更导致测试失败**
|
||||
- **问题:** 修改 UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment] 后，测试代码期望字符串列表
|
||||
- **解决:** 批量更新测试以访问 .content 属性（使用 sed 批量替换）
|
||||
- **影响:** 需要更新 tests/test_semantic_engine.py 中的多处代码
|
||||
|
||||
## Decisions Made
|
||||
|
||||
**Architecture Decision: UnifiedSegmentationStrategy 返回 VideoSegment 列表**
|
||||
- **Rationale:** Plan 13-07 要求支持带标题的分段输出，返回 VideoSegment 对象保留标题信息，转换为纯文本会丢失
|
||||
- **Trade-off:** 破坏了 Plan 13-06 的 API 返回类型（List[str] → List[VideoSegment]），但这是实现功能的必要变更
|
||||
- **Mitigation:** 添加代理方法到 SemanticSegmentationEngine，更新所有测试以适配新 API
|
||||
|
||||
**Design Decision: SRTProcessor 步骤顺序调整**
|
||||
- **Rationale:** LLM 语义分段需要 metadata 构建 Prompt（视频类型、标题等），必须在分段之前加载元数据
|
||||
- **Impact:** 改变了原有的步骤顺序（原来是：解析 → 分段 → 纠错 → 加载元数据 → 生成 Markdown）
|
||||
- **New order:** 解析 → 加载元数据 → 分段 → 纠错 → 生成 Markdown
|
||||
|
||||
**Implementation Decision: SRTOutput 仍返回纯文本列表**
|
||||
- **Rationale:** SRTOutput Schema 定义 paragraphs 为 List[str]，修改 Schema 会影响更多代码
|
||||
- **Approach:** 在返回 SRTOutput 之前将 VideoSegment 转换为纯文本（提取 .content 字段）
|
||||
- **Benefit:** 保持 SRTOutput Schema 不变，减少影响范围
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
**Ready for Phase 14 (鲁棒性):**
|
||||
- SRTProcessor 集成完成，支持 LLM 语义分段和规则分段
|
||||
- 自动降级机制已实现，AI 不可用时优雅降级
|
||||
- 全面的测试覆盖（集成测试 9 个，语义引擎测试 39 个）
|
||||
- 无阻塞问题或关注点
|
||||
|
||||
**Architecture Benefits:**
|
||||
- 配置开关控制（enable_semantic_segmentation）便于逐步迁移
|
||||
- 自动降级确保鲁棒性
|
||||
- 向后兼容保持现有行为
|
||||
- 策略模式支持未来扩展
|
||||
|
||||
**Known Limitations:**
|
||||
- LLM 分段结果不稳定（多次运行可能不同）- Phase 14 需要实现温度控制
|
||||
- 长视频（>1小时）可能超出上下文窗口 - Phase 14 需要实现分块处理
|
||||
- 无缓存机制 - Phase 14 需要实现结果缓存
|
||||
|
||||
---
|
||||
*Phase: 13-llm*
|
||||
*Plan: 07*
|
||||
*Completed: 2026-03-21*
|
||||
Reference in New Issue
Block a user