--- phase: 02-platform-skills plan: 01 type: execute wave: 1 depends_on: [] files_modified: - kb_pipeline/link_parser.py - kb_pipeline/idempotency.py - kb_pipeline/__init__.py - tests/test_pipeline/test_link_parser.py - tests/test_pipeline/test_idempotency.py autonomous: true requirements: [PLAT-01, PLAT-02, PLAT-03] user_setup: [] must_haves: truths: - "输入B站视频URL,解析出 platform=bilibili, type=video, id=BVxxx" - "输入知乎专栏URL,解析出 platform=zhihu, type=zhuanlan, id=xxx" - "输入b23.tv短链,自动解析到真实URL并提取ID" - "输入无法识别的URL,返回None和错误信息(不抛异常)" - "幂等检查能正确判断文件是否已存在" artifacts: - path: "kb_pipeline/link_parser.py" provides: "统一链接解析器" exports: ["LinkParser", "ParsedLink"] - path: "kb_pipeline/idempotency.py" provides: "幂等检查机制" exports: ["IdempotencyChecker"] - path: "tests/test_pipeline/test_link_parser.py" provides: "LinkParser 单元测试" min_lines: 50 - path: "tests/test_pipeline/test_idempotency.py" provides: "幂等检查单元测试" min_lines: 30 key_links: - from: "kb_pipeline/link_parser.py" to: "kb_pipeline/config.py" via: "get_settings for base_path" pattern: "from.*config.*import" - from: "kb_pipeline/idempotency.py" to: "kb_pipeline/link_parser.py" via: "ParsedLink for content_id" pattern: "from.*link_parser.*import" --- 创建统一的 LinkParser 类和幂等检查机制,为后续平台适配器提供链接解析和重复处理防护能力。 Purpose: 为所有平台提供统一的 URL 解析入口,支持 B站/知乎的内容类型识别和 ID 提取,同时提供幂等检查避免重复处理。 Output: LinkParser 类、ParsedLink 数据类、IdempotencyChecker 类 @/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md @/home/starmen1265/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/02-platform-skills/02-CONTEXT.md @.planning/phases/02-platform-skills/02-RESEARCH.md @.planning/phases/02-platform-skills/02-VALIDATION.md From kb_pipeline/schemas.py: ```python class StepInput(BaseModel): content_id: str = Field(..., min_length=1, description="内容唯一标识") platform: Literal["bilibili", "zhihu"] = Field(..., description="平台名称") class StepStatus(str, Enum): PENDING = "pending" COMPLETED = "completed" FAILED = "failed" ``` From kb_pipeline/config.py: ```python class KBSettings(BaseSettings): base_path: str = Field(..., description="知识库根目录") archive_path: str = Field(..., description="归档目录") @lru_cache def get_settings() -> KBSettings: """获取配置单例""" ``` From kb_pipeline/errors.py: ```python class KBError(Exception): def __init__(self, message, error_type, step_name, input_summary, suggestion, cause=None) class ValidationError(KBError): def __init__(self, step_name, input_summary, validation_errors=None, cause=None) ``` From kb_pipeline/retry.py: ```python retry_on_network_error = retry( stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10), retry=retry_if_exception_type(NetworkError), ) ``` From kb-link-parser/link-parser.py: ```python class LinkParser: BILIBILI_VIDEO_PATTERN = re.compile(r'https?://(?:www\.)?bilibili\.com/video/(BV[a-zA-Z0-9]+)') BILIBILI_ARTICLE_PATTERN = re.compile(r'https?://(?:www\.)?bilibili\.com/(?:article|read)/([a-zA-Z0-9]+)') ZHIHAU_ANSWER_PATTERN = re.compile(r'https?://(?:www\.)?zhihu\.com/question/\d+/answer/(\d+)') ZHIHAU_ZHUANLAN_PATTERN = re.compile(r'https?://zhuanlan\.zhihu\.com/p/(\d+)') def resolve_shortlink(self, url: str) -> Optional[str]: # 使用 requests.head(url, allow_redirects=True, timeout=10) def parse(self, url: str) -> Dict[str, Any]: # 返回 {platform, type, id, original_url, resolved_url, success, error} ``` Task 1: Write interface contracts and tests for LinkParser kb_pipeline/link_parser.py, tests/test_pipeline/test_link_parser.py - Test 1: 解析B站视频URL返回正确的 ParsedLink (platform=bilibili, type=video, id=BVxxx) - Test 2: 解析B站专栏URL返回正确的 ParsedLink (platform=bilibili, type=article, id=cvxxx) - Test 3: 解析知乎问答URL返回正确的 ParsedLink (platform=zhihu, type=answer, id=xxx) - Test 4: 解析知乎专栏URL返回正确的 ParsedLink (platform=zhihu, type=zhuanlan, id=xxx) - Test 5: 解析b23.tv短链返回解析后的真实URL - Test 6: 无法识别的URL返回None(不抛异常) 1. 创建 kb_pipeline/link_parser.py: - 定义 ParsedLink dataclass,包含: platform, content_type, content_id, original_url, resolved_url - 定义 LinkParser 类,实现 parse(url) -> Optional[ParsedLink] - 支持 URL 类型: * B站视频: bilibili.com/video/BVxxx * B站专栏: bilibili.com/read/cvxxx * 知乎问答: zhihu.com/question/xxx/answer/xxx * 知乎专栏: zhuanlan.zhihu.com/p/xxx - 短链支持: b23.tv 使用 requests.head(allow_redirects=True) 解析 - 错误处理: 无法识别返回 None,不抛异常 2. 创建 tests/test_pipeline/test_link_parser.py: - 使用 pytest 和 parametrize 测试各种 URL 格式 - Mock 短链解析以避免真实网络请求 - 覆盖所有 CONTEXT.md 定义的支持 URL 类型 pytest tests/test_pipeline/test_link_parser.py -x -v LinkParser 能正确解析所有支持的 URL 类型,无法识别的 URL 返回 None Task 2: Write interface contracts and tests for IdempotencyChecker kb_pipeline/idempotency.py, tests/test_pipeline/test_idempotency.py - Test 1: 文件存在时 check_exists 返回 (True, [file_list]) - Test 2: 文件不存在时 check_exists 返回 (False, []) - Test 3: 部分文件存在时返回正确的存在文件列表 1. 创建 kb_pipeline/idempotency.py: - 定义 IdempotencyChecker 类 - 实现 check_exists(output_dir: Path, content_id: str, platform: str) -> Tuple[bool, list[Path]] - 检查文件: * 元数据: {platform}_{content_id}_metadata.json * 内容: {platform}_{content_id}_content.md - 使用 pathlib.Path 进行路径操作 2. 创建 tests/test_pipeline/test_idempotency.py: - 使用 tmp_path fixture 创建临时文件 - 测试文件存在/不存在的各种场景 - 测试部分文件存在的情况 pytest tests/test_pipeline/test_idempotency.py -x -v IdempotencyChecker 能正确检查文件是否存在,返回准确的文件列表 Task 3: Update package exports kb_pipeline/__init__.py 更新 kb_pipeline/__init__.py,导出新模块: ```python from .link_parser import LinkParser, ParsedLink from .idempotency import IdempotencyChecker __all__ = [ # ... existing exports ... "LinkParser", "ParsedLink", "IdempotencyChecker", ] ``` python -c "from kb_pipeline import LinkParser, ParsedLink, IdempotencyChecker; print('OK')" LinkParser, ParsedLink, IdempotencyChecker 可从 kb_pipeline 包根导入 - 所有测试通过: `pytest tests/test_pipeline/test_link_parser.py tests/test_pipeline/test_idempotency.py -v` - 导入测试: `python -c "from kb_pipeline import LinkParser, IdempotencyChecker"` - 覆盖率: `pytest tests/test_pipeline/ --cov=kb_pipeline --cov-report=term-missing` - [ ] LinkParser 正确解析 B站视频/专栏、知乎问答/专栏 URL - [ ] b23.tv 短链自动解析到真实 URL - [ ] 无法识别的 URL 返回 None(不抛异常) - [ ] IdempotencyChecker 正确检查文件存在性 - [ ] 所有测试通过(至少 10 个测试用例) - [ ] 公共 API 可从 kb_pipeline 包根导入 After completion, create `.planning/phases/02-platform-skills/02-01-SUMMARY.md`