kb-skills/.planning/phases/14-robustness-cost/14-00-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
14-robustness-cost	00	execute	0		tests/test_semantic_robustness.py tests/conftest.py tests/test_srt_processor.py	true	ROBUST-01 ROBUST-02 ROBUST-03 ROBUST-04 COST-01 COST-02 COST-03	truths artifacts key_links 所有计划任务都有对应的自动化测试 测试框架配置正确，可以运行测试 Mock fixtures 可用，模拟 LLM 客户端行为 path provides exports tests/test_semantic_robustness.py 鲁棒性和成本控制测试套件 test_* path provides exports tests/conftest.py 共享测试 fixtures mock_ai_client sample_segments sample_metadata path provides exports tests/test_srt_processor.py SRTProcessor 集成测试 test_semantic_segmentation_toggle from to via pattern test_semantic_robustness.py conftest.py pytest fixtures @pytest.fixture

创建阶段 14 的测试基础设施，覆盖所有鲁棒性和成本控制功能。

Purpose: 确保所有计划任务都有对应的自动化测试，遵循 TDD 原则。 Output: 完整的测试套件，包括鲁棒性测试、成本控制测试、集成测试。

<execution_context> @/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md @/home/starmen1265/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/phases/14-robustness-cost/14-CONTEXT.md @.planning/phases/14-robustness-cost/14-RESEARCH.md @.planning/phases/14-robustness-cost/14-VALIDATION.md @.planning/ROADMAP.md @.planning/REQUIREMENTS.md @.planning/STATE.md

@tests/conftest.py @tests/test_semantic_engine.py @kb_workflow/srt/semantic_engine.py

From tests/conftest.py (existing fixtures):

import pytest
from unittest.mock import Mock, MagicMock
from kb_workflow.srt.parser import Segment

@pytest.fixture
def mock_ai_client():
    """Mock AI client"""
    client = Mock()
    client.api_key = "test_key"
    client.generate = Mock(return_value="Test response")
    return client

@pytest.fixture
def sample_segments():
    """示例字幕分段"""
    return [
        Segment(index=1, start_time=0.0, end_time=1.0, text="第一句"),
        Segment(index=2, start_time=1.0, end_time=2.0, text="第二句"),
    ]

@pytest.fixture
def sample_metadata():
    """示例视频元数据"""
    return {
        "title": "测试视频",
        "duration": 120,
        "category": "知识"
    }

From 14-VALIDATION.md (Wave 0 Requirements):

# Wave 0 Requirements
- [ ] tests/test_semantic_robustness.py — 覆盖 ROBUST-01, ROBUST-02, ROBUST-03, ROBUST-04, COST-01, COST-02
- [ ] tests/test_srt_processor.py — 覆盖 COST-03（配置开关）
- [ ] tests/conftest.py — 共享 fixtures（mock LLM client, 样本字幕数据）
- [ ] 框架安装: 无需安装，pytest 已配置

# Mock Strategy
使用 unittest.mock.patch 模拟 SimpleAIClient.generate()，测试不同失败场景（网络错误、解析失败、超时等）

From 14-RESEARCH.md (Test Requirements):

Req ID	Behavior	Test Type	Automated Command
ROBUST-01	LLM 调用失败时自动降级	unit	test_llm_failure_fallback
ROBUST-02	解析失败时重试或降级	unit	test_parse_failure_retry
ROBUST-03	AI 不可用时使用规则分段	unit	test_ai_unavailable_fallback
ROBUST-04	降级日志记录	unit	test_fallback_logging
COST-01	长文本分块处理	unit	test_long_text_chunking
COST-02	缓存命中避免重复调用	unit	test_cache_hit
COST-03	配置开关控制	integration	test_semantic_segmentation_toggle

Task 1: 创建 test_semantic_robustness.py 测试文件 tests/test_semantic_robustness.py 创建 tests/test_semantic_robustness.py，包含所有鲁棒性和成本控制测试:

1. 导入依赖:
   ```python
   import pytest
   import json
   from unittest.mock import Mock, patch, MagicMock
   from pathlib import Path
   from requests.exceptions import ConnectionError, Timeout, HTTPError

   from kb_workflow.srt.semantic_engine import (
       UnifiedSegmentationStrategy,
       SemanticSegmentationEngine,
       VideoSegment,
       SegmentationResult
   )
   from kb_workflow.srt.cache import SemanticCache, compute_content_hash
   ```

2. 创建测试类和方法（遵循 14-VALIDATION.md 的测试映射）:

**TestLLMFailureFallback (ROBUST-01)**:
- test_llm_failure_fallback: 模拟 LLM 调用失败（网络错误），验证降级到规则分段
- test_recoverable_error_retry: 模拟可恢复错误（ConnectionError），验证重试 1 次
- test_non_recoverable_error_no_retry: 模拟不可恢复错误（HTTPError 400），验证直接降级

**TestParseFailureRetry (ROBUST-02, ROBUST-04)**:
- test_parse_failure_retry: 模拟 JSON 解析失败（JSONDecodeError），验证重试后降级
- test_validation_error_no_retry: 模拟 Pydantic 验证失败（ValidationError），验证直接降级
- test_fallback_logging: 验证降级时记录 warning 级别日志，包含错误类型和原因

**TestAIAvailabilityFallback (ROBUST-03)**:
- test_ai_unavailable_fallback: 模拟 AI 客户端未配置（api_key 为空），验证使用规则分段
- test_ai_client_none: 模拟 AI 客户端初始化失败，验证使用规则分段

**TestLongTextChunking (COST-01)**:
- test_long_text_chunking: 使用超过 5000 字符的文本，验证分块处理
- test_chunk_sentence_boundary: 验证分块在句子边界（标点符号后）分割
- test_chunk_index_correction: 验证 VideoSegment.start_index/end_index 相对于原文

**TestCacheHit (COST-02)**:
- test_cache_hit: 第一次调用 LLM，第二次调用从缓存读取
- test_cache_key_hash: 验证缓存键使用 SHA256 哈希
- test_cache_miss_content_change: 内容变化时缓存未命中
- test_cache_file_format: 验证缓存文件格式（JSON + version 字段）

3. 使用 Mock 模拟 LLM 客户端:
   ```python
   @pytest.fixture
   def mock_ai_client_with_failure():
       """模拟失败的 AI 客户端"""
       client = Mock()
       client.api_key = "test_key"
       client.generate = Mock(side_effect=ConnectionError("Network error"))
       return client

   @pytest.fixture
   def mock_ai_client_with_parse_error():
       """模拟解析失败的 AI 客户端"""
       client = Mock()
       client.api_key = "test_key"
       # 第一次返回无效 JSON，第二次返回有效 JSON
       client.generate = Mock(side_effect=[
           "Invalid JSON",
           '{"video_type": "知识类", "segments": [], "actual_segment_count": 0}'
       ])
       return client
   ```

4. 示例测试实现:
   ```python
   def test_llm_failure_fallback(sample_segments, sample_metadata):
       """测试 LLM 调用失败时降级到规则分段"""
       strategy = UnifiedSegmentationStrategy(
           ai_client=mock_ai_client_with_failure()
       )

       result = strategy.segment(sample_segments, sample_metadata)

       # 验证降级到规则分段（返回 VideoSegment 列表）
       assert isinstance(result, list)
       assert all(isinstance(seg, VideoSegment) for seg in result)
       # 规则分段没有标题
       assert all(seg.title is None for seg in result)

   def test_recoverable_error_retry(sample_segments, sample_metadata):
       """测试可恢复错误重试 1 次"""
       client = Mock()
       client.api_key = "test_key"
       # 第一次失败，第二次成功
       client.generate = Mock(side_effect=[
           ConnectionError("Network error"),
           '{"video_type": "知识类", "segments": [], "actual_segment_count": 0}'
       ])

       strategy = UnifiedSegmentationStrategy(ai_client=client)
       result = strategy.segment(sample_segments, sample_metadata)

       # 验证重试了 2 次
       assert client.generate.call_count == 2
   ```

确保所有测试都是 RED 状态（功能尚未实现）。

pytest tests/test_semantic_robustness.py -v --tb=no - test_semantic_robustness.py 文件创建，包含所有测试 - 所有测试都是 RED 状态（预期失败） - 测试覆盖所有需求（ROBUST-01 到 COST-03） - Mock fixtures 正确模拟 LLM 客户端行为 Task 2: 更新 conftest.py 添加新 fixtures tests/conftest.py 更新 tests/conftest.py，添加阶段 14 需要的新 fixtures:

1. 添加长文本样本（用于分块测试）:
   ```python
   @pytest.fixture
   def long_text_sample():
       """长文本样本（>5000 字符，用于分块测试）"""
       # 生成一个超过 5000 字符的文本，包含多个句子
       sentences = ["这是一个测试句子。" * 10] * 15  # 约 6000 字符
       return " ".join(sentences)
   ```

2. 添加模拟失败的 AI 客户端 fixtures:
   ```python
   @pytest.fixture
   def mock_ai_client_network_error():
       """模拟网络错误的 AI 客户端"""
       client = Mock()
       client.api_key = "test_key"
       client.generate = Mock(side_effect=ConnectionError("Network error"))
       return client

   @pytest.fixture
   def mock_ai_client_timeout():
       """模拟超时的 AI 客户端"""
       client = Mock()
       client.api_key = "test_key"
       client.generate = Mock(side_effect=Timeout("Request timeout"))
       return client

   @pytest.fixture
   def mock_ai_client_rate_limit():
       """模拟限流的 AI 客户端"""
       response = Mock()
       response.status_code = 429
       client = Mock()
       client.api_key = "test_key"
       client.generate = Mock(side_effect=HTTPError(response=response))
       return client

   @pytest.fixture
   def mock_ai_client_api_error():
       """模拟 API 错误的 AI 客户端（不可恢复）"""
       response = Mock()
       response.status_code = 400
       client = Mock()
       client.api_key = "test_key"
       client.generate = Mock(side_effect=HTTPError(response=response))
       return client

   @pytest.fixture
   def mock_ai_client_parse_error():
       """模拟解析失败的 AI 客户端"""
       client = Mock()
       client.api_key = "test_key"
       # 第一次返回无效 JSON
       client.generate = Mock(return_value="Invalid JSON response")
       return client

   @pytest.fixture
   def mock_ai_client_success():
       """模拟成功的 AI 客户端"""
       client = Mock()
       client.api_key = "test_key"
       client.generate = Mock(return_value='''{
           "video_type": "知识类",
           "segments": [
               {
                   "title": "测试标题",
                   "content": "测试内容",
                   "start_index": 0,
                   "end_index": 10
               }
           ],
           "actual_segment_count": 1
       }''')
       return client
   ```

3. 添加缓存相关的 fixtures:
   ```python
   @pytest.fixture
   def temp_cache_dir(tmp_path):
       """临时缓存目录"""
       cache_dir = tmp_path / "cache" / "semantic"
       return cache_dir

   @pytest.fixture
   def semantic_cache(temp_cache_dir):
       """语义分段缓存实例"""
       from kb_workflow.srt.cache import SemanticCache
       return SemanticCache(cache_dir=temp_cache_dir)
   ```

4. 添加长字幕分段样本:
   ```python
   @pytest.fixture
   def long_subtitle_segments():
       """长字幕分段样本（>5000 字符）"""
       segments = []
       for i in range(100):
           segments.append(Segment(
               index=i + 1,
               start_time=i * 1.0,
               end_time=(i + 1) * 1.0,
               text=f"这是第 {i + 1} 句字幕内容。" * 5
           ))
       return segments
   ```

pytest tests/conftest.py --collect-only - conftest.py 添加所有需要的 fixtures - Fixtures 包括: 模拟失败的 AI 客户端、长文本样本、缓存目录 - 所有 fixtures 可以正确导入和使用 Task 3: 在 test_srt_processor.py 中添加配置开关测试 tests/test_srt_processor.py 在 tests/test_srt_processor.py 中添加配置开关测试（COST-03）:

1. 如果文件不存在，创建基础文件:
   ```python
   import pytest
   from unittest.mock import Mock, patch
   from pathlib import Path

   from kb_workflow.srt.processor import SRTProcessor
   from kb_workflow.srt.parser import SRTParser
   ```

2. 添加 TestSemanticSegmentationToggle 测试类:
   ```python
   class TestSemanticSegmentationToggle:
       """测试语义分段配置开关 (COST-03)"""

       def test_semantic_segmentation_enabled(self, sample_srt_file):
           """测试启用语义分段"""
           with patch('kb_workflow.srt.semantic_engine.SimpleAIClient') as mock_client:
               mock_client.return_value = Mock(api_key="test_key")

               processor = SRTProcessor(
                   config={},
                   enable_semantic_segmentation=True
               )

               # 验证创建了 SemanticSegmentationEngine
               assert processor.semantic_engine is not None
               assert processor.segmenter is None

       def test_semantic_segmentation_disabled(self, sample_srt_file):
           """测试禁用语义分段（默认）"""
           processor = SRTProcessor(
               config={},
               enable_semantic_segmentation=False
           )

           # 验证使用规则分段器
           assert processor.semantic_engine is None
           assert processor.segmenter is not None

       def test_semantic_segmentation_with_cache(self, sample_srt_file):
           """测试启用语义分段和缓存"""
           with patch('kb_workflow.srt.semantic_engine.SimpleAIClient') as mock_client:
               mock_client.return_value = Mock(api_key="test_key")

               processor = SRTProcessor(
                   config={},
                   enable_semantic_segmentation=True,
                   enable_cache=True
               )

               # 验证创建了缓存
               assert processor.semantic_engine is not None
               # 验证策略使用了缓存
               assert processor.semantic_engine.strategy.cache is not None

       def test_semantic_segmentation_without_cache(self, sample_srt_file):
           """测试启用语义分段但禁用缓存"""
           with patch('kb_workflow.srt.semantic_engine.SimpleAIClient') as mock_client:
               mock_client.return_value = Mock(api_key="test_key")

               processor = SRTProcessor(
                   config={},
                   enable_semantic_segmentation=True,
                   enable_cache=False
               )

               # 验证没有创建缓存
               assert processor.semantic_engine is not None
               assert processor.semantic_engine.strategy.cache is None

       @pytest.fixture
       def sample_srt_file(self, tmp_path):
           """创建示例 SRT 文件"""
           srt_content = """1
   00:00:00,000 --> 00:00:01,000
   第一句字幕

   2
   00:00:01,000 --> 00:00:02,000
   第二句字幕
   """
           srt_file = tmp_path / "test.srt"
           srt_file.write_text(srt_content, encoding='utf-8')
           return srt_file
   ```

3. 确保测试是 RED 状态（功能尚未实现）。

pytest tests/test_srt_processor.py::test_semantic_segmentation_toggle -v --tb=no - test_srt_processor.py 添加配置开关测试 - 测试覆盖启用/禁用语义分段、启用/禁用缓存 - 所有测试都是 RED 状态（预期失败） ## 整体验证标准

1. 测试文件创建: test_semantic_robustness.py 包含所有鲁棒性和成本控制测试
2. Fixtures 更新: conftest.py 添加所有需要的 fixtures
3. 集成测试: test_srt_processor.py 添加配置开关测试
4. 测试状态: 所有测试都是 RED 状态（功能尚未实现）

运行测试验证文件结构:

# 验证测试文件存在
ls tests/test_semantic_robustness.py tests/conftest.py tests/test_srt_processor.py

# 验证测试可以收集
python3 -m pytest tests/test_semantic_robustness.py --collect-only
python3 -m pytest tests/test_srt_processor.py::test_semantic_segmentation_toggle --collect-only

# 验证测试是 RED 状态
python3 -m pytest tests/test_semantic_robustness.py -v --tb=no

<success_criteria>

1. test_semantic_robustness.py 文件创建，包含 6 个测试类
2. conftest.py 添加 10+ 个新 fixtures
3. test_srt_processor.py 添加 TestSemanticSegmentationToggle 测试类
4. 所有测试都是 RED 状态（预期失败）
5. 测试覆盖所有需求（ROBUST-01 到 COST-03） </success_criteria>

执行完成后创建 `.planning/phases/14-robustness-cost/14-00-SUMMARY.md`