---
phase: 01-foundation
plan: 02
type: tdd
wave: 1
depends_on: []
files_modified: [kb_pipeline/__init__.py, kb_pipeline/schemas.py, tests/test_pipeline/__init__.py, tests/test_pipeline/conftest.py, tests/test_pipeline/test_schemas.py]
autonomous: true
requirements: [PIPE-01, PIPE-03, VAL-01, REL-03]
user_setup: []
must_haves:
truths:
- "每个步骤输出通过 Pydantic Schema 验证，格式错误阻止进入下一步"
- "所有步骤的 Schema 集中在一个模块，便于维护"
- "步骤执行前验证输入，执行后验证输出"
artifacts:
- path: "kb_pipeline/schemas.py"
provides: "核心 Schema 定义"
min_lines: 100
exports: ["StepInput", "StepOutput", "StepResult", "StepStatus", "StepRecord", "WorkflowState", "StepDefinition"]
- path: "kb_pipeline/__init__.py"
provides: "包初始化和公共接口导出"
- path: "tests/test_pipeline/test_schemas.py"
provides: "Schema 验证测试"
min_lines: 80
key_links:
- from: "StepOutput"
to: "Pydantic BaseModel"
via: "继承和验证"
pattern: "class StepOutput.*BaseModel"
- from: "model_validate()"
to: "Schema 验证"
via: "Pydantic 验证方法"
pattern: "model_validate|ValidationError"
---
定义工作流管道的核心 Schema，包括 StepInput、StepOutput、StepResult、WorkflowState 等，使用 Pydantic v2 确保类型安全和自动验证。
Purpose: 建立统一的数据格式规范，所有后续步骤和状态管理都基于这些 Schema
Output: kb_pipeline/schemas.py + 完整的单元测试
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
From CONTEXT.md decisions:
- Schema 格式: Pydantic 模型 — 自带验证和序列化，类型安全
- 验证范围: 验证输入和输出 — 步骤执行前验证输入，执行后验证输出
- Schema 定义位置: 集中定义 — 所有步骤的 Schema 集中在一个模块
From RESEARCH.md Pattern 2: Pydantic Schema Registry:
```python
from pydantic import BaseModel, Field
from typing import Optional, Literal
from datetime import datetime
class StepInput(BaseModel):
"""所有步骤输入的基类"""
content_id: str = Field(..., description="内容唯一标识，如 BV号")
platform: Literal["bilibili", "zhihu"] = Field(..., description="平台名称")
class StepOutput(BaseModel):
"""所有步骤输出的基类"""
success: bool = Field(..., description="步骤是否成功")
error_message: Optional[str] = Field(None, description="错误信息")
completed_at: datetime = Field(default_factory=datetime.now)
```
Core Pipeline Schemas
kb_pipeline/schemas.py, tests/test_pipeline/test_schemas.py
- StepInput 定义所有步骤的公共输入字段 (content_id, platform)
- StepOutput 定义所有步骤的公共输出字段 (success, error_message, completed_at)
- StepStatus 枚举定义步骤状态 (PENDING, RUNNING, COMPLETED, FAILED, SKIPPED)
- StepRecord 记录单个步骤的执行状态
- WorkflowState 管理整个工作流的状态
- StepDefinition 定义步骤的前置条件、输出格式、成功标准
- 所有 Schema 支持 Pydantic model_validate() 验证
1. 创建 kb_pipeline/ 目录和 __init__.py
2. 创建 tests/test_pipeline/ 目录和 conftest.py
3. 在 schemas.py 中定义所有 Schema:
- StepStatus (Enum)
- StepInput (BaseModel)
- StepOutput (BaseModel)
- StepResult (BaseModel) - 包含输入输出和状态
- StepRecord (BaseModel) - 步骤执行记录
- WorkflowState (BaseModel) - 工作流状态
- StepDefinition (BaseModel) - 步骤定义
Task 1: Create package structure and write failing tests
kb_pipeline/__init__.py, tests/test_pipeline/__init__.py, tests/test_pipeline/conftest.py, tests/test_pipeline/test_schemas.py
- 测试 StepInput 必须包含 content_id 和 platform
- 测试 StepOutput 必须包含 success 字段
- 测试 StepStatus 包含所有必要状态
- 测试 WorkflowState 包含 steps 字典
- 测试 Pydantic 验证失败时抛出 ValidationError
1. 创建目录结构:
- mkdir -p kb_pipeline tests/test_pipeline
2. 创建 kb_pipeline/__init__.py:
```python
"""KB Pipeline - 工作流管道基础设施"""
from .schemas import (
StepInput, StepOutput, StepResult,
StepStatus, StepRecord, WorkflowState, StepDefinition
)
__all__ = [...]
```
3. 创建 tests/test_pipeline/__init__.py (空文件)
4. 创建 tests/test_pipeline/conftest.py:
- 添加通用 fixtures (tmp_content_dir, sample_workflow_state)
5. 创建 tests/test_pipeline/test_schemas.py，包含测试类:
- TestStepInput: 测试必填字段和验证
- TestStepOutput: 测试输出字段和默认值
- TestStepStatus: 测试枚举值
- TestStepRecord: 测试步骤记录
- TestWorkflowState: 测试工作流状态
- TestSchemaValidation: 测试 Pydantic 验证
pytest tests/test_pipeline/test_schemas.py -v --collect-only 2>/dev/null | grep "test_" | wc -l | xargs -I {} test {} -ge 12
测试文件包含至少 12 个测试用例，所有测试当前失败（RED 阶段）
Task 2: Implement core schemas
kb_pipeline/schemas.py
- StepInput 验证 content_id 非空
- StepOutput 默认 completed_at 为当前时间
- StepStatus 包含 PENDING, RUNNING, COMPLETED, FAILED, SKIPPED
- WorkflowState.steps 是 Dict[str, StepRecord]
- 所有模型支持 model_dump_json() 序列化
实现 kb_pipeline/schemas.py:
```python
from pydantic import BaseModel, Field
from typing import Optional, Dict, Literal, Any
from datetime import datetime
from enum import Enum
class StepStatus(str, Enum):
"""步骤执行状态"""
PENDING = "pending"
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
SKIPPED = "skipped"
class StepInput(BaseModel):
"""所有步骤输入的基类"""
content_id: str = Field(..., min_length=1, description="内容唯一标识")
platform: Literal["bilibili", "zhihu"] = Field(..., description="平台名称")
class StepOutput(BaseModel):
"""所有步骤输出的基类"""
success: bool = Field(..., description="步骤是否成功")
error_message: Optional[str] = Field(None, description="错误信息")
completed_at: datetime = Field(default_factory=datetime.now)
class StepResult(BaseModel):
"""步骤执行结果"""
step_name: str
input_data: StepInput
output_data: Optional[StepOutput] = None
status: StepStatus = StepStatus.PENDING
class StepRecord(BaseModel):
"""单个步骤的执行记录"""
step_name: str
status: StepStatus
started_at: Optional[datetime] = None
completed_at: Optional[datetime] = None
error_message: Optional[str] = None
output_summary: Optional[str] = None
class WorkflowState(BaseModel):
"""工作流状态 - 存储在 workflow_state.json"""
schema_version: str = Field(default="1.0")
content_id: str = Field(..., description="内容唯一标识")
platform: str = Field(..., description="平台名称")
source_url: str = Field(..., description="原始链接")
created_at: datetime = Field(default_factory=datetime.now)
updated_at: datetime = Field(default_factory=datetime.now)
steps: Dict[str, StepRecord] = Field(default_factory=dict)
def is_step_completed(self, step_name: str) -> bool:
record = self.steps.get(step_name)
return record is not None and record.status == StepStatus.COMPLETED
def mark_step_started(self, step_name: str) -> None:
self.steps[step_name] = StepRecord(
step_name=step_name,
status=StepStatus.RUNNING,
started_at=datetime.now()
)
self.updated_at = datetime.now()
def mark_step_completed(self, step_name: str, output_summary: str = None) -> None:
if step_name in self.steps:
self.steps[step_name].status = StepStatus.COMPLETED
self.steps[step_name].completed_at = datetime.now()
self.steps[step_name].output_summary = output_summary
else:
self.steps[step_name] = StepRecord(
step_name=step_name,
status=StepStatus.COMPLETED,
completed_at=datetime.now(),
output_summary=output_summary
)
self.updated_at = datetime.now()
def mark_step_failed(self, step_name: str, error_message: str) -> None:
if step_name in self.steps:
self.steps[step_name].status = StepStatus.FAILED
self.steps[step_name].error_message = error_message
self.steps[step_name].completed_at = datetime.now()
else:
self.steps[step_name] = StepRecord(
step_name=step_name,
status=StepStatus.FAILED,
error_message=error_message,
completed_at=datetime.now()
)
self.updated_at = datetime.now()
class StepDefinition(BaseModel):
"""步骤定义 - 描述步骤的前置条件、输出格式、成功标准"""
step_name: str
description: str
input_schema: type[StepInput]
output_schema: type[StepOutput]
dependencies: list[str] = Field(default_factory=list)
```
注意: WorkflowState 的方法实现先写在 schemas.py 中，后续会考虑移到 state.py
pytest tests/test_pipeline/test_schemas.py -v --tb=short
所有测试通过（GREEN 阶段）
Task 3: Refactor and verify JSON serialization
kb_pipeline/schemas.py, tests/test_pipeline/test_schemas.py
- model_dump_json() 生成有效的 JSON
- model_validate() 从 JSON 正确反序列化
- JSON Schema 可导出 (用于文档)
1. 添加 JSON 序列化测试:
- TestJSONSerialization: 测试 model_dump_json 和 model_validate
2. 验证所有 Schema 支持:
- model_dump() -> dict
- model_dump_json() -> str
- model_validate(data) -> instance
3. 确保 WorkflowState 的 steps 字典正确序列化/反序列化
pytest tests/test_pipeline/test_schemas.py -v --tb=short -x
所有测试通过，JSON 序列化正常工作
1. 运行 `pytest tests/test_pipeline/test_schemas.py -v` 全部通过
2. 验证 StepInput 缺少 content_id 时抛出 ValidationError
3. 验证 WorkflowState.model_dump_json() 生成有效 JSON
4. 验证 WorkflowState.model_validate() 可从 JSON 恢复
- 所有核心 Schema 定义完成
- Pydantic 验证正常工作
- JSON 序列化/反序列化正常
- 测试覆盖率 > 80%