starmen1265
bd40f4965b
Change depends_on from [01-02] to ["02"] for consistency with plan frontmatter format. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
499 lines
17 KiB
Markdown
499 lines
17 KiB
Markdown
---
|
|
phase: 01-foundation
|
|
plan: 03
|
|
type: tdd
|
|
wave: 2
|
|
depends_on: ["02"]
|
|
files_modified: [kb_pipeline/errors.py, kb_pipeline/config.py, kb_pipeline/retry.py, tests/test_pipeline/test_errors.py, tests/test_pipeline/test_config.py, tests/test_pipeline/test_retry.py]
|
|
autonomous: true
|
|
requirements: [REL-02, MNT-02, MNT-03]
|
|
user_setup:
|
|
- service: pydantic-settings
|
|
why: "配置管理需要读取 ~/.kb_env 文件"
|
|
env_vars:
|
|
- name: KB_BASE_PATH
|
|
source: "~/.kb_env 文件中定义"
|
|
- name: KB_ARCHIVE_PATH
|
|
source: "~/.kb_env 文件中定义"
|
|
dashboard_config: []
|
|
|
|
must_haves:
|
|
truths:
|
|
- "所有环境变量集中在 ~/.kb_env，启动时验证必需变量"
|
|
- "错误信息包含步骤名、输入摘要、失败原因、可操作建议"
|
|
- "网络错误自动重试3次，指数退避"
|
|
artifacts:
|
|
- path: "kb_pipeline/errors.py"
|
|
provides: "结构化错误类型"
|
|
min_lines: 60
|
|
exports: ["KBError", "NetworkError", "ValidationError", "SystemError", "AuthError", "ErrorType"]
|
|
- path: "kb_pipeline/config.py"
|
|
provides: "统一配置管理"
|
|
min_lines: 40
|
|
exports: ["KBSettings", "get_settings"]
|
|
- path: "kb_pipeline/retry.py"
|
|
provides: "重试装饰器"
|
|
min_lines: 30
|
|
exports: ["retry_on_network_error"]
|
|
- path: "tests/test_pipeline/test_errors.py"
|
|
provides: "错误处理测试"
|
|
- path: "tests/test_pipeline/test_config.py"
|
|
provides: "配置管理测试"
|
|
- path: "tests/test_pipeline/test_retry.py"
|
|
provides: "重试逻辑测试"
|
|
key_links:
|
|
- from: "KBSettings"
|
|
to: "~/.kb_env"
|
|
via: "pydantic-settings env_file"
|
|
pattern: "SettingsConfigDict.*env_file"
|
|
- from: "KBError.to_dict()"
|
|
to: "错误信息输出"
|
|
via: "结构化序列化"
|
|
pattern: "step_name.*input_summary.*suggestion"
|
|
- from: "retry_on_network_error"
|
|
to: "网络请求"
|
|
via: "装饰器"
|
|
pattern: "@retry.*stop_after_attempt.*3"
|
|
---
|
|
|
|
<objective>
|
|
实现结构化错误处理、统一配置管理和网络重试机制，为工作流提供可靠的错误恢复和配置验证能力。
|
|
|
|
Purpose: 确保错误信息包含足够上下文便于调试，配置集中管理便于维护，网络错误自动恢复
|
|
Output: kb_pipeline/errors.py, config.py, retry.py + 完整的单元测试
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
|
|
<interfaces>
|
|
<!-- Dependencies from 01-02 schemas -->
|
|
|
|
From kb_pipeline/schemas.py:
|
|
```python
|
|
class StepStatus(str, Enum):
|
|
PENDING = "pending"
|
|
RUNNING = "running"
|
|
COMPLETED = "completed"
|
|
FAILED = "failed"
|
|
SKIPPED = "skipped"
|
|
```
|
|
|
|
From CONTEXT.md decisions:
|
|
- 错误分类: 网络错误 (403/404/500/超时), 验证错误, 系统错误, 认证/配额错误
|
|
- 重试策略: 网络错误自动重试3次，指数退避
|
|
- 错误信息: 步骤名、输入摘要、错误类型、失败原因、可操作建议
|
|
- 配置文件位置: ~/.kb_env
|
|
- 配置库: pydantic-settings
|
|
|
|
From RESEARCH.md Pattern 3 & 4:
|
|
```python
|
|
# pydantic-settings configuration
|
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
|
|
|
class KBSettings(BaseSettings):
|
|
model_config = SettingsConfigDict(
|
|
env_file='~/.kb_env',
|
|
env_file_encoding='utf-8',
|
|
env_prefix='KB_',
|
|
extra='ignore'
|
|
)
|
|
base_path: str = Field(..., description="知识库根目录")
|
|
archive_path: str = Field(..., description="归档目录")
|
|
```
|
|
|
|
```python
|
|
# Structured error handling
|
|
class ErrorType(str, Enum):
|
|
NETWORK = "network"
|
|
VALIDATION = "validation"
|
|
SYSTEM = "system"
|
|
AUTH = "auth"
|
|
|
|
class KBError(Exception):
|
|
def __init__(self, message, error_type, step_name, input_summary, suggestion, cause=None):
|
|
...
|
|
```
|
|
</interfaces>
|
|
</context>
|
|
|
|
<feature>
|
|
<name>Error Handler & Config Manager</name>
|
|
<files>kb_pipeline/errors.py, kb_pipeline/config.py, kb_pipeline/retry.py, tests/test_pipeline/test_errors.py, tests/test_pipeline/test_config.py, tests/test_pipeline/test_retry.py</files>
|
|
<behavior>
|
|
- KBError 包含 step_name, input_summary, error_type, message, suggestion
|
|
- NetworkError, ValidationError, SystemError, AuthError 继承自 KBError
|
|
- KBSettings 从 ~/.kb_env 加载配置，缺失必需变量时报错
|
|
- retry_on_network_error 装饰器自动重试网络错误3次
|
|
- 错误可序列化为 dict 用于日志和状态记录
|
|
</behavior>
|
|
<implementation>
|
|
1. 实现 kb_pipeline/errors.py:
|
|
- ErrorType 枚举
|
|
- KBError 基类 (包含 to_dict 方法)
|
|
- NetworkError, ValidationError, SystemError, AuthError 子类
|
|
|
|
2. 实现 kb_pipeline/config.py:
|
|
- KBSettings (pydantic-settings BaseSettings)
|
|
- get_settings() 单例函数
|
|
|
|
3. 实现 kb_pipeline/retry.py:
|
|
- retry_on_network_error 装饰器 (使用 tenacity)
|
|
</implementation>
|
|
</feature>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 1: Write failing tests for error handling</name>
|
|
<files>tests/test_pipeline/test_errors.py</files>
|
|
<behavior>
|
|
- Test KBError contains all required fields
|
|
- Test KBError.to_dict() returns correct dict
|
|
- Test NetworkError includes status_code
|
|
- Test each error type has appropriate suggestion
|
|
</behavior>
|
|
<action>
|
|
创建 tests/test_pipeline/test_errors.py:
|
|
|
|
```python
|
|
import pytest
|
|
from kb_pipeline.errors import (
|
|
KBError, NetworkError, ValidationError, SystemError, AuthError, ErrorType
|
|
)
|
|
|
|
class TestKBError:
|
|
def test_kberror_contains_all_fields(self):
|
|
error = KBError(
|
|
message="Test error",
|
|
error_type=ErrorType.NETWORK,
|
|
step_name="test_step",
|
|
input_summary="input_id=123",
|
|
suggestion="Check your network"
|
|
)
|
|
assert error.step_name == "test_step"
|
|
assert error.input_summary == "input_id=123"
|
|
assert error.suggestion == "Check your network"
|
|
|
|
def test_kberror_to_dict(self):
|
|
error = KBError(...)
|
|
result = error.to_dict()
|
|
assert "step_name" in result
|
|
assert "error_type" in result
|
|
assert "suggestion" in result
|
|
|
|
class TestNetworkError:
|
|
def test_network_error_includes_status_code(self):
|
|
error = NetworkError(
|
|
step_name="fetch",
|
|
input_summary="url=http://example.com",
|
|
status_code=403
|
|
)
|
|
assert error.status_code == 403
|
|
assert "403" in error.suggestion
|
|
|
|
class TestErrorHierarchy:
|
|
def test_network_error_is_kberror(self):
|
|
assert issubclass(NetworkError, KBError)
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_errors.py -v --collect-only 2>/dev/null | grep "test_" | wc -l | xargs -I {} test {} -ge 8</automated>
|
|
</verify>
|
|
<done>测试文件包含至少 8 个测试用例，所有测试当前失败（RED 阶段）</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 2: Implement error handling</name>
|
|
<files>kb_pipeline/errors.py</files>
|
|
<behavior>
|
|
- ErrorType enum with NETWORK, VALIDATION, SYSTEM, AUTH
|
|
- KBError base class with to_dict method
|
|
- NetworkError with status_code and auto-suggestion
|
|
- ValidationError for schema failures
|
|
- SystemError for file/memory/dependency issues
|
|
- AuthError for API quota/token issues
|
|
</behavior>
|
|
<action>
|
|
实现 kb_pipeline/errors.py:
|
|
|
|
```python
|
|
from enum import Enum
|
|
from typing import Optional, Any
|
|
|
|
class ErrorType(str, Enum):
|
|
"""错误类型分类"""
|
|
NETWORK = "network"
|
|
VALIDATION = "validation"
|
|
SYSTEM = "system"
|
|
AUTH = "auth"
|
|
|
|
class KBError(Exception):
|
|
"""KB 工作流基础错误"""
|
|
def __init__(
|
|
self,
|
|
message: str,
|
|
error_type: ErrorType,
|
|
step_name: str,
|
|
input_summary: str,
|
|
suggestion: str,
|
|
cause: Optional[Exception] = None
|
|
):
|
|
super().__init__(message)
|
|
self.error_type = error_type
|
|
self.step_name = step_name
|
|
self.input_summary = input_summary
|
|
self.suggestion = suggestion
|
|
self.cause = cause
|
|
|
|
def to_dict(self) -> dict:
|
|
return {
|
|
"error_type": self.error_type.value,
|
|
"step_name": self.step_name,
|
|
"input_summary": self.input_summary,
|
|
"message": str(self),
|
|
"suggestion": self.suggestion
|
|
}
|
|
|
|
class NetworkError(KBError):
|
|
"""网络错误 (403/404/500/超时)"""
|
|
def __init__(self, step_name: str, input_summary: str, status_code: int = None, cause: Exception = None):
|
|
suggestion = f"检查网络连接，状态码: {status_code}" if status_code else "检查网络连接"
|
|
super().__init__(
|
|
message=f"网络请求失败: {status_code or 'unknown'}",
|
|
error_type=ErrorType.NETWORK,
|
|
step_name=step_name,
|
|
input_summary=input_summary,
|
|
suggestion=suggestion,
|
|
cause=cause
|
|
)
|
|
self.status_code = status_code
|
|
|
|
class ValidationError(KBError):
|
|
"""验证错误 (Schema 验证失败)"""
|
|
def __init__(self, step_name: str, input_summary: str, validation_errors: list = None, cause: Exception = None):
|
|
super().__init__(
|
|
message="Schema 验证失败",
|
|
error_type=ErrorType.VALIDATION,
|
|
step_name=step_name,
|
|
input_summary=input_summary,
|
|
suggestion="检查输入数据格式是否符合要求",
|
|
cause=cause
|
|
)
|
|
self.validation_errors = validation_errors or []
|
|
|
|
class SystemError(KBError):
|
|
"""系统错误 (文件系统/内存/依赖缺失)"""
|
|
def __init__(self, step_name: str, input_summary: str, system_message: str = None, cause: Exception = None):
|
|
super().__init__(
|
|
message=f"系统错误: {system_message or 'unknown'}",
|
|
error_type=ErrorType.SYSTEM,
|
|
step_name=step_name,
|
|
input_summary=input_summary,
|
|
suggestion="检查系统资源和依赖是否满足要求",
|
|
cause=cause
|
|
)
|
|
|
|
class AuthError(KBError):
|
|
"""认证/配额错误 (API配额/Cookie过期/Token失效)"""
|
|
def __init__(self, step_name: str, input_summary: str, auth_type: str = None, cause: Exception = None):
|
|
super().__init__(
|
|
message=f"认证失败: {auth_type or 'unknown'}",
|
|
error_type=ErrorType.AUTH,
|
|
step_name=step_name,
|
|
input_summary=input_summary,
|
|
suggestion="检查 API Token 或 Cookie 是否有效",
|
|
cause=cause
|
|
)
|
|
self.auth_type = auth_type
|
|
```
|
|
|
|
注意: SystemError 与 Python 内置同名，考虑命名空间或使用别名
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_errors.py -v --tb=short</automated>
|
|
</verify>
|
|
<done>所有测试通过（GREEN 阶段）</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 3: Write and implement configuration management</name>
|
|
<files>kb_pipeline/config.py, tests/test_pipeline/test_config.py</files>
|
|
<behavior>
|
|
- KBSettings loads from ~/.kb_env
|
|
- Required fields: base_path, archive_path
|
|
- Optional fields: git_root, git_user, git_email
|
|
- Secret fields: hf_token, zhihu_cookie (from env vars)
|
|
- get_settings() returns singleton
|
|
- Missing required field raises ValidationError
|
|
</behavior>
|
|
<action>
|
|
1. 创建 tests/test_pipeline/test_config.py:
|
|
- Test KBSettings loads from env file
|
|
- Test required fields validation
|
|
- Test get_settings returns singleton
|
|
|
|
2. 实现 kb_pipeline/config.py:
|
|
|
|
```python
|
|
from pydantic import Field, SecretStr
|
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
|
from functools import lru_cache
|
|
|
|
class KBSettings(BaseSettings):
|
|
"""KB 工作流配置"""
|
|
model_config = SettingsConfigDict(
|
|
env_file=str(Path.home() / '.kb_env'),
|
|
env_file_encoding='utf-8',
|
|
env_prefix='KB_',
|
|
extra='ignore'
|
|
)
|
|
|
|
# 必需配置
|
|
base_path: str = Field(..., description="知识库根目录")
|
|
archive_path: str = Field(..., description="归档目录")
|
|
|
|
# 可选配置
|
|
git_root: str = Field(default="", description="Git 仓库根目录")
|
|
git_user: str = Field(default="", description="Git 用户名")
|
|
git_email: str = Field(default="", description="Git 邮箱")
|
|
|
|
# 敏感信息（从环境变量读取）
|
|
hf_token: SecretStr = Field(default=SecretStr(""), description="HuggingFace Token")
|
|
zhihu_cookie: SecretStr = Field(default=SecretStr(""), description="知乎 Cookie")
|
|
|
|
@lru_cache
|
|
def get_settings() -> KBSettings:
|
|
"""获取配置单例"""
|
|
return KBSettings()
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_config.py -v --tb=short</automated>
|
|
</verify>
|
|
<done>所有测试通过，配置管理正常工作</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 4: Write and implement retry mechanism</name>
|
|
<files>kb_pipeline/retry.py, tests/test_pipeline/test_retry.py</files>
|
|
<behavior>
|
|
- retry_on_network_error decorator retries 3 times
|
|
- Uses exponential backoff (2s, 4s, 8s max)
|
|
- Only retries on NetworkError
|
|
- Logs retry attempts
|
|
- Reraises after max attempts
|
|
</behavior>
|
|
<action>
|
|
1. 创建 tests/test_pipeline/test_retry.py:
|
|
- Test retry succeeds on second attempt
|
|
- Test retry exhausts attempts and raises
|
|
- Test non-NetworkError is not retried
|
|
- Test exponential backoff timing
|
|
|
|
2. 实现 kb_pipeline/retry.py:
|
|
|
|
```python
|
|
from tenacity import (
|
|
retry,
|
|
stop_after_attempt,
|
|
wait_exponential,
|
|
retry_if_exception_type,
|
|
before_sleep_log
|
|
)
|
|
import logging
|
|
|
|
from .errors import NetworkError
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
retry_on_network_error = retry(
|
|
stop=stop_after_attempt(3),
|
|
wait=wait_exponential(multiplier=1, min=2, max=10),
|
|
retry=retry_if_exception_type(NetworkError),
|
|
before_sleep=before_sleep_log(logger, logging.WARNING),
|
|
reraise=True
|
|
)
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_pipeline/test_retry.py -v --tb=short</automated>
|
|
</verify>
|
|
<done>所有测试通过，重试机制正常工作</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 5: Update package exports</name>
|
|
<files>kb_pipeline/__init__.py</files>
|
|
<behavior>
|
|
- All public APIs exported from package
|
|
- Clean import interface
|
|
</behavior>
|
|
<action>
|
|
更新 kb_pipeline/__init__.py 导出:
|
|
|
|
```python
|
|
"""KB Pipeline - 工作流管道基础设施"""
|
|
|
|
from .schemas import (
|
|
StepInput, StepOutput, StepResult,
|
|
StepStatus, StepRecord, WorkflowState, StepDefinition
|
|
)
|
|
from .state import WorkflowStateManager
|
|
from .errors import (
|
|
KBError, NetworkError, ValidationError,
|
|
SystemError, AuthError, ErrorType
|
|
)
|
|
from .config import KBSettings, get_settings
|
|
from .retry import retry_on_network_error
|
|
|
|
__all__ = [
|
|
# Schemas
|
|
"StepInput", "StepOutput", "StepResult",
|
|
"StepStatus", "StepRecord", "WorkflowState", "StepDefinition",
|
|
# State
|
|
"WorkflowStateManager",
|
|
# Errors
|
|
"KBError", "NetworkError", "ValidationError",
|
|
"SystemError", "AuthError", "ErrorType",
|
|
# Config
|
|
"KBSettings", "get_settings",
|
|
# Retry
|
|
"retry_on_network_error",
|
|
]
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>python -c "from kb_pipeline import WorkflowStateManager, KBSettings, retry_on_network_error" && echo "OK"</automated>
|
|
</verify>
|
|
<done>所有公共 API 可从包根导入</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
1. 运行 `pytest tests/test_pipeline/test_errors.py -v` 全部通过
|
|
2. 运行 `pytest tests/test_pipeline/test_config.py -v` 全部通过
|
|
3. 运行 `pytest tests/test_pipeline/test_retry.py -v` 全部通过
|
|
4. 验证错误 to_dict() 包含所有必要字段
|
|
5. 验证配置缺失必需字段时抛出 ValidationError
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- 结构化错误类型完整，包含上下文信息
|
|
- 配置从 ~/.kb_env 加载，缺失必需字段报错
|
|
- 网络错误自动重试，指数退避
|
|
- 所有测试通过，覆盖率 > 80%
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/01-foundation/01-03-SUMMARY.md`
|
|
</output>
|