kb-skills/.planning/phases/14-robustness-cost/14-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created

phase	slug	status	nyquist_compliant	wave_0_complete	created
14	robustness-cost	draft	true	true	2026-03-23

Phase 14 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	pytest 7.x
Config file	pytest.ini (已存在)
Quick run command	python3 -m pytest tests/test_semantic_robustness.py -v -x
Full suite command	python3 -m pytest tests/ -v
Estimated runtime	~5 seconds

---

Sampling Rate

• After every task commit: Run python3 -m pytest tests/test_semantic_robustness.py -v -x
• After every plan wave: Run python3 -m pytest tests/ -v
• Before /gsd:verify-work: Full suite must be green
• Max feedback latency: 10 seconds

---

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Test Type	Automated Command	File Exists	Status
14-01-01	01	1	ROBUST-01	unit	pytest tests/test_semantic_robustness.py::test_llm_failure_fallback -x	❌ W0	⬜ pending
14-02-01	02	1	ROBUST-02	unit	pytest tests/test_semantic_robustness.py::test_parse_failure_retry -x	❌ W0	⬜ pending
14-02-02	02	1	ROBUST-04	unit	pytest tests/test_semantic_robustness.py::test_fallback_logging -x	❌ W0	⬜ pending
14-03-01	03	1	ROBUST-03	unit	pytest tests/test_semantic_robustness.py::test_ai_unavailable_fallback -x	❌ W0	⬜ pending
14-04-01	04	1	COST-01	unit	pytest tests/test_semantic_robustness.py::test_long_text_chunking -x	❌ W0	⬜ pending
14-05-01	05	1	COST-02	unit	pytest tests/test_semantic_robustness.py::test_cache_hit -x	❌ W0	⬜ pending
14-05-02	05	1	COST-03	integration	pytest tests/test_srt_processor.py::test_semantic_segmentation_toggle -x	❌ W0	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

---

Wave 0 Requirements

• tests/test_semantic_robustness.py — 覆盖 ROBUST-01, ROBUST-02, ROBUST-03, ROBUST-04, COST-01, COST-02
• tests/test_srt_processor.py — 覆盖 COST-03（配置开关）
• tests/conftest.py — 共享 fixtures（mock LLM client, 样本字幕数据）
• 框架安装: 无需安装，pytest 已配置

Mock Strategy: 使用 unittest.mock.patch 模拟 SimpleAIClient.generate()，测试不同失败场景（网络错误、解析失败、超时等）

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
无	-	-	所有阶段行为都有自动化验证

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies
• Sampling continuity: no 3 consecutive tasks without automated verify
• Wave 0 covers all MISSING references
• No watch-mode flags
• Feedback latency < 10s
• nyquist_compliant: true set in frontmatter

Approval: pending