320 lines
12 KiB
Markdown
320 lines
12 KiB
Markdown
---
|
|
phase: 01-foundation
|
|
plan: 02
|
|
type: tdd
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified: [kb_pipeline/__init__.py, kb_pipeline/schemas.py, tests/test_pipeline/__init__.py, tests/test_pipeline/conftest.py, tests/test_pipeline/test_schemas.py]
|
|
autonomous: true
|
|
requirements: [PIPE-01, PIPE-03, VAL-01, REL-03]
|
|
user_setup: []
|
|
|
|
must_haves:
|
|
truths:
|
|
- "每个步骤输出通过 Pydantic Schema 验证，格式错误阻止进入下一步"
|
|
- "所有步骤的 Schema 集中在一个模块，便于维护"
|
|
- "步骤执行前验证输入，执行后验证输出"
|
|
artifacts:
|
|
- path: "kb_pipeline/schemas.py"
|
|
provides: "核心 Schema 定义"
|
|
min_lines: 100
|
|
exports: ["StepInput", "StepOutput", "StepResult", "StepStatus", "StepRecord", "WorkflowState", "StepDefinition"]
|
|
- path: "kb_pipeline/__init__.py"
|
|
provides: "包初始化和公共接口导出"
|
|
- path: "tests/test_pipeline/test_schemas.py"
|
|
provides: "Schema 验证测试"
|
|
min_lines: 80
|
|
key_links:
|
|
- from: "StepOutput"
|
|
to: "Pydantic BaseModel"
|
|
via: "继承和验证"
|
|
pattern: "class StepOutput.*BaseModel"
|
|
- from: "model_validate()"
|
|
to: "Schema 验证"
|
|
via: "Pydantic 验证方法"
|
|
pattern: "model_validate|ValidationError"
|
|
---
|
|
|
|
<objective>
|
|
定义工作流管道的核心 Schema，包括 StepInput、StepOutput、StepResult、WorkflowState 等，使用 Pydantic v2 确保类型安全和自动验证。
|
|
|
|
Purpose: 建立统一的数据格式规范，所有后续步骤和状态管理都基于这些 Schema
|
|
Output: kb_pipeline/schemas.py + 完整的单元测试
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
|
|
<interfaces>
|
|
<!-- No external dependencies - this is the foundation layer -->
|
|
<!-- Following patterns from RESEARCH.md and CONTEXT.md decisions -->
|
|
|
|
From CONTEXT.md decisions:
|
|
- Schema 格式: Pydantic 模型 — 自带验证和序列化，类型安全
|
|
- 验证范围: 验证输入和输出 — 步骤执行前验证输入，执行后验证输出
|
|
- Schema 定义位置: 集中定义 — 所有步骤的 Schema 集中在一个模块
|
|
|
|
From RESEARCH.md Pattern 2: Pydantic Schema Registry:
|
|
```python
|
|
from pydantic import BaseModel, Field
|
|
from typing import Optional, Literal
|
|
from datetime import datetime
|
|
|
|
class StepInput(BaseModel):
|
|
"""所有步骤输入的基类"""
|
|
content_id: str = Field(..., description="内容唯一标识，如 BV号")
|
|
platform: Literal["bilibili", "zhihu"] = Field(..., description="平台名称")
|
|
|
|
class StepOutput(BaseModel):
|
|
"""所有步骤输出的基类"""
|
|
success: bool = Field(..., description="步骤是否成功")
|
|
error_message: Optional[str] = Field(None, description="错误信息")
|
|
completed_at: datetime = Field(default_factory=datetime.now)
|
|
```
|
|
</interfaces>
|
|
</context>
|
|
|
|
<feature>
|
|
<name>Core Pipeline Schemas</name>
|
|
<files>kb_pipeline/schemas.py, tests/test_pipeline/test_schemas.py</files>
|
|
<behavior>
|
|
- StepInput 定义所有步骤的公共输入字段 (content_id, platform)
|
|
- StepOutput 定义所有步骤的公共输出字段 (success, error_message, completed_at)
|
|
- StepStatus 枚举定义步骤状态 (PENDING, RUNNING, COMPLETED, FAILED, SKIPPED)
|
|
- StepRecord 记录单个步骤的执行状态
|
|
- WorkflowState 管理整个工作流的状态
|
|
- StepDefinition 定义步骤的前置条件、输出格式、成功标准
|
|
- 所有 Schema 支持 Pydantic model_validate() 验证
|
|
</behavior>
|
|
<implementation>
|
|
1. 创建 kb_pipeline/ 目录和 __init__.py
|
|
2. 创建 tests/test_pipeline/ 目录和 conftest.py
|
|
3. 在 schemas.py 中定义所有 Schema:
|
|
- StepStatus (Enum)
|
|
- StepInput (BaseModel)
|
|
- StepOutput (BaseModel)
|
|
- StepResult (BaseModel) - 包含输入输出和状态
|
|
- StepRecord (BaseModel) - 步骤执行记录
|
|
- WorkflowState (BaseModel) - 工作流状态
|
|
- StepDefinition (BaseModel) - 步骤定义
|
|
</implementation>
|
|
</feature>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 1: Create package structure and write failing tests</name>
|
|
<files>kb_pipeline/__init__.py, tests/test_pipeline/__init__.py, tests/test_pipeline/conftest.py, tests/test_pipeline/test_schemas.py</files>
|
|
<behavior>
|
|
- 测试 StepInput 必须包含 content_id 和 platform
|
|
- 测试 StepOutput 必须包含 success 字段
|
|
- 测试 StepStatus 包含所有必要状态
|
|
- 测试 WorkflowState 包含 steps 字典
|
|
- 测试 Pydantic 验证失败时抛出 ValidationError
|
|
</behavior>
|
|
<action>
|
|
1. 创建目录结构:
|
|
- mkdir -p kb_pipeline tests/test_pipeline
|
|
|
|
2. 创建 kb_pipeline/__init__.py:
|
|
```python
|
|
"""KB Pipeline - 工作流管道基础设施"""
|
|
from .schemas import (
|
|
StepInput, StepOutput, StepResult,
|
|
StepStatus, StepRecord, WorkflowState, StepDefinition
|
|
)
|
|
__all__ = [...]
|
|
```
|
|
|
|
3. 创建 tests/test_pipeline/__init__.py (空文件)
|
|
|
|
4. 创建 tests/test_pipeline/conftest.py:
|
|
- 添加通用 fixtures (tmp_content_dir, sample_workflow_state)
|
|
|
|
5. 创建 tests/test_pipeline/test_schemas.py，包含测试类:
|
|
- TestStepInput: 测试必填字段和验证
|
|
- TestStepOutput: 测试输出字段和默认值
|
|
- TestStepStatus: 测试枚举值
|
|
- TestStepRecord: 测试步骤记录
|
|
- TestWorkflowState: 测试工作流状态
|
|
- TestSchemaValidation: 测试 Pydantic 验证
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_schemas.py -v --collect-only 2>/dev/null | grep "test_" | wc -l | xargs -I {} test {} -ge 12</automated>
|
|
</verify>
|
|
<done>测试文件包含至少 12 个测试用例，所有测试当前失败（RED 阶段）</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 2: Implement core schemas</name>
|
|
<files>kb_pipeline/schemas.py</files>
|
|
<behavior>
|
|
- StepInput 验证 content_id 非空
|
|
- StepOutput 默认 completed_at 为当前时间
|
|
- StepStatus 包含 PENDING, RUNNING, COMPLETED, FAILED, SKIPPED
|
|
- WorkflowState.steps 是 Dict[str, StepRecord]
|
|
- 所有模型支持 model_dump_json() 序列化
|
|
</behavior>
|
|
<action>
|
|
实现 kb_pipeline/schemas.py:
|
|
|
|
```python
|
|
from pydantic import BaseModel, Field
|
|
from typing import Optional, Dict, Literal, Any
|
|
from datetime import datetime
|
|
from enum import Enum
|
|
|
|
class StepStatus(str, Enum):
|
|
"""步骤执行状态"""
|
|
PENDING = "pending"
|
|
RUNNING = "running"
|
|
COMPLETED = "completed"
|
|
FAILED = "failed"
|
|
SKIPPED = "skipped"
|
|
|
|
class StepInput(BaseModel):
|
|
"""所有步骤输入的基类"""
|
|
content_id: str = Field(..., min_length=1, description="内容唯一标识")
|
|
platform: Literal["bilibili", "zhihu"] = Field(..., description="平台名称")
|
|
|
|
class StepOutput(BaseModel):
|
|
"""所有步骤输出的基类"""
|
|
success: bool = Field(..., description="步骤是否成功")
|
|
error_message: Optional[str] = Field(None, description="错误信息")
|
|
completed_at: datetime = Field(default_factory=datetime.now)
|
|
|
|
class StepResult(BaseModel):
|
|
"""步骤执行结果"""
|
|
step_name: str
|
|
input_data: StepInput
|
|
output_data: Optional[StepOutput] = None
|
|
status: StepStatus = StepStatus.PENDING
|
|
|
|
class StepRecord(BaseModel):
|
|
"""单个步骤的执行记录"""
|
|
step_name: str
|
|
status: StepStatus
|
|
started_at: Optional[datetime] = None
|
|
completed_at: Optional[datetime] = None
|
|
error_message: Optional[str] = None
|
|
output_summary: Optional[str] = None
|
|
|
|
class WorkflowState(BaseModel):
|
|
"""工作流状态 - 存储在 workflow_state.json"""
|
|
schema_version: str = Field(default="1.0")
|
|
content_id: str = Field(..., description="内容唯一标识")
|
|
platform: str = Field(..., description="平台名称")
|
|
source_url: str = Field(..., description="原始链接")
|
|
created_at: datetime = Field(default_factory=datetime.now)
|
|
updated_at: datetime = Field(default_factory=datetime.now)
|
|
steps: Dict[str, StepRecord] = Field(default_factory=dict)
|
|
|
|
def is_step_completed(self, step_name: str) -> bool:
|
|
record = self.steps.get(step_name)
|
|
return record is not None and record.status == StepStatus.COMPLETED
|
|
|
|
def mark_step_started(self, step_name: str) -> None:
|
|
self.steps[step_name] = StepRecord(
|
|
step_name=step_name,
|
|
status=StepStatus.RUNNING,
|
|
started_at=datetime.now()
|
|
)
|
|
self.updated_at = datetime.now()
|
|
|
|
def mark_step_completed(self, step_name: str, output_summary: str = None) -> None:
|
|
if step_name in self.steps:
|
|
self.steps[step_name].status = StepStatus.COMPLETED
|
|
self.steps[step_name].completed_at = datetime.now()
|
|
self.steps[step_name].output_summary = output_summary
|
|
else:
|
|
self.steps[step_name] = StepRecord(
|
|
step_name=step_name,
|
|
status=StepStatus.COMPLETED,
|
|
completed_at=datetime.now(),
|
|
output_summary=output_summary
|
|
)
|
|
self.updated_at = datetime.now()
|
|
|
|
def mark_step_failed(self, step_name: str, error_message: str) -> None:
|
|
if step_name in self.steps:
|
|
self.steps[step_name].status = StepStatus.FAILED
|
|
self.steps[step_name].error_message = error_message
|
|
self.steps[step_name].completed_at = datetime.now()
|
|
else:
|
|
self.steps[step_name] = StepRecord(
|
|
step_name=step_name,
|
|
status=StepStatus.FAILED,
|
|
error_message=error_message,
|
|
completed_at=datetime.now()
|
|
)
|
|
self.updated_at = datetime.now()
|
|
|
|
class StepDefinition(BaseModel):
|
|
"""步骤定义 - 描述步骤的前置条件、输出格式、成功标准"""
|
|
step_name: str
|
|
description: str
|
|
input_schema: type[StepInput]
|
|
output_schema: type[StepOutput]
|
|
dependencies: list[str] = Field(default_factory=list)
|
|
```
|
|
|
|
注意: WorkflowState 的方法实现先写在 schemas.py 中，后续会考虑移到 state.py
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_schemas.py -v --tb=short</automated>
|
|
</verify>
|
|
<done>所有测试通过（GREEN 阶段）</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 3: Refactor and verify JSON serialization</name>
|
|
<files>kb_pipeline/schemas.py, tests/test_pipeline/test_schemas.py</files>
|
|
<behavior>
|
|
- model_dump_json() 生成有效的 JSON
|
|
- model_validate() 从 JSON 正确反序列化
|
|
- JSON Schema 可导出 (用于文档)
|
|
</behavior>
|
|
<action>
|
|
1. 添加 JSON 序列化测试:
|
|
- TestJSONSerialization: 测试 model_dump_json 和 model_validate
|
|
|
|
2. 验证所有 Schema 支持:
|
|
- model_dump() -> dict
|
|
- model_dump_json() -> str
|
|
- model_validate(data) -> instance
|
|
|
|
3. 确保 WorkflowState 的 steps 字典正确序列化/反序列化
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_schemas.py -v --tb=short -x</automated>
|
|
</verify>
|
|
<done>所有测试通过，JSON 序列化正常工作</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
1. 运行 `pytest tests/test_pipeline/test_schemas.py -v` 全部通过
|
|
2. 验证 StepInput 缺少 content_id 时抛出 ValidationError
|
|
3. 验证 WorkflowState.model_dump_json() 生成有效 JSON
|
|
4. 验证 WorkflowState.model_validate() 可从 JSON 恢复
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- 所有核心 Schema 定义完成
|
|
- Pydantic 验证正常工作
|
|
- JSON 序列化/反序列化正常
|
|
- 测试覆盖率 > 80%
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/01-foundation/01-02-SUMMARY.md`
|
|
</output>
|