10 KiB
10 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, user_setup, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | user_setup | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 02-platform-skills | 03 | execute | 2 |
|
|
true |
|
|
|
Purpose: 为知乎问答和专栏提供统一的内容提取接口,支持 Cookie 认证和错误处理。 Output: ZhihuAdapter 实现类
<execution_context> @/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md @/home/starmen1265/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/02-platform-skills/02-CONTEXT.md @.planning/phases/02-platform-skills/02-RESEARCH.mdFrom kb_pipeline/platforms/base.py (Plan 02):
class PlatformAdapter(ABC):
@abstractmethod
def get_content_type(self, url: str) -> str: pass
@abstractmethod
async def fetch_metadata(self, content_id: str, content_type: str) -> Dict[str, Any]: pass
@abstractmethod
async def fetch_content(self, content_id: str, content_type: str) -> str: pass
@abstractmethod
def get_output_filename(self, content_id: str, content_type: str, suffix: str) -> str: pass
From kb_pipeline/config.py:
class KBSettings(BaseSettings):
zhihu_cookie: SecretStr = Field(default=SecretStr(""), description="知乎 Cookie")
From kb_pipeline/errors.py:
class AuthError(KBError):
def __init__(self, step_name, input_summary, auth_type=None, cause=None):
# suggestion: "检查 API Token 或 Cookie 是否有效"
From kb_pipeline/link_parser.py (Plan 01):
@dataclass
class ParsedLink:
platform: str
content_type: str # "answer" | "zhuanlan"
content_id: str
<existing_code>
From kb-zhihu-metadata/zhihu_extractor.py:
class ZhihuPlaywrightExtractor:
URL_PATTERNS = {
'zvideo': r'zhihu\.com/zvideo/(\d+)',
'zhuanlan': r'zhuanlan\.zhihu\.com/p/(\d+)',
'answer': r'zhihu\.com/question/\d+/answer/(\d+)',
'question': r'zhihu\.com/question/(\d+)',
}
def __init__(self, headless=True, timeout=30, cookie=None):
self.cookie = cookie
self.browser = PlaywrightBrowser(headless=headless, timeout=timeout)
def extract(self, url: str) -> Optional[Dict[str, Any]]:
# 返回 {type, title, author, content, content_markdown, publish_time, ...}
def _extract_answer_content(self) -> Dict[str, Any]: ...
def _extract_zhuanlan_content(self) -> Dict[str, Any]: ...
</existing_code>
Task 1: Implement ZhihuAdapter for answer content kb_pipeline/platforms/zhihu.py, tests/test_platforms/test_zhihu.py - Test 1: fetch_metadata("xxx", "answer") 返回正确的问答元数据 - Test 2: fetch_content("xxx", "answer") 返回 Markdown 格式的回答内容 - Test 3: Cookie 缺失时返回 AuthError - Test 4: get_output_filename 返回正确格式 "zhihu_xxx_metadata.json" 创建 kb_pipeline/platforms/zhihu.py: ```python from kb_pipeline.platforms.base import PlatformAdapter from kb_pipeline.errors import AuthError, NetworkError from kb_pipeline.config import get_settings from kb_zhihu_metadata.zhihu_extractor import ZhihuPlaywrightExtractorclass ZhihuAdapter(PlatformAdapter):
PLATFORM = "zhihu"
def __init__(self, headless: bool = True, timeout: int = 30):
self.headless = headless
self.timeout = timeout
self._cookie = None
@property
def cookie(self) -> str:
if self._cookie is None:
settings = get_settings()
self._cookie = settings.zhihu_cookie.get_secret_value()
return self._cookie
def get_content_type(self, url: str) -> str:
# 从 URL 识别 answer 或 zhuanlan
if "answer" in url:
return "answer"
elif "zhuanlan" in url:
return "zhuanlan"
return "unknown"
async def fetch_metadata(self, content_id: str, content_type: str) -> Dict[str, Any]:
if not self.cookie:
raise AuthError(
step_name="zhihu_metadata",
input_summary=f"content_id={content_id}",
auth_type="zhihu_cookie",
)
# 使用 ZhihuPlaywrightExtractor 提取
pass
async def fetch_content(self, content_id: str, content_type: str) -> str:
# 返回 Markdown 格式内容
pass
def get_output_filename(self, content_id: str, content_type: str, suffix: str) -> str:
return f"zhihu_{content_id}_{suffix}"
```
创建 tests/test_platforms/test_zhihu.py:
- Mock ZhihuPlaywrightExtractor 的返回值
- 测试 Cookie 缺失时的 AuthError
- 测试问答内容提取
pytest tests/test_platforms/test_zhihu.py::test_answer_content -x -v
ZhihuAdapter 能正确提取知乎问答内容,Cookie 缺失时返回 AuthError
Task 2: Implement ZhihuAdapter for zhuanlan content
kb_pipeline/platforms/zhihu.py, tests/test_platforms/test_zhihu.py
- Test 1: fetch_metadata("xxx", "zhuanlan") 返回正确的专栏元数据
- Test 2: fetch_content("xxx", "zhuanlan") 返回 Markdown 格式的文章内容
- Test 3: 检测到登录页面时返回 AuthError 并提示更新 Cookie
扩展 kb_pipeline/platforms/zhihu.py:
1. 实现 zhuanlan 类型的 fetch_metadata 和 fetch_content
2. 添加登录页面检测逻辑:
```python
def _detect_login_page(self, result: Dict) -> bool:
"""检测是否被重定向到登录页面"""
if not result:
return False
title = result.get("title", "")
content = result.get("content", "")
return "登录" in title or len(content) < 100
```
3. 登录页面检测时抛出 AuthError:
```python
if self._detect_login_page(result):
raise AuthError(
step_name="zhihu_content",
input_summary=f"content_id={content_id}",
auth_type="zhihu_cookie",
cause=Exception("Cookie 可能已过期,请更新 KB_ZHIHU_COOKIE")
)
```
4. 扩展测试覆盖专栏场景
pytest tests/test_platforms/test_zhihu.py::test_zhuanlan_content -x -v
ZhihuAdapter 能正确提取知乎专栏内容,检测登录页面时返回 AuthError
Task 3: Update package exports
kb_pipeline/__init__.py, kb_pipeline/platforms/__init__.py
1. 更新 kb_pipeline/platforms/__init__.py:
```python
from .base import PlatformAdapter
from .bilibili import BilibiliAdapter
from .zhihu import ZhihuAdapter
__all__ = ["PlatformAdapter", "BilibiliAdapter", "ZhihuAdapter"]
```
2. 更新 kb_pipeline/__init__.py:
```python
from .platforms import PlatformAdapter, BilibiliAdapter, ZhihuAdapter
__all__ = [
# ... existing exports ...
"PlatformAdapter",
"BilibiliAdapter",
"ZhihuAdapter",
]
```
python -c "from kb_pipeline import ZhihuAdapter; print('OK')"
ZhihuAdapter 可从 kb_pipeline 包根导入
- 所有测试通过: `pytest tests/test_platforms/test_zhihu.py -v`
- 导入测试: `python -c "from kb_pipeline import ZhihuAdapter"`
- 覆盖率: `pytest tests/test_platforms/ --cov=kb_pipeline/platforms --cov-report=term-missing`
<success_criteria>
- ZhihuAdapter 正确提取知乎问答内容
- ZhihuAdapter 正确提取知乎专栏内容
- Cookie 缺失时返回 AuthError
- 检测到登录页面时返回 AuthError 并提示更新 Cookie
- 内容保存为 Markdown 格式,包含 YAML Frontmatter
- 所有测试通过(至少 6 个测试用例) </success_criteria>