303 lines
10 KiB
Markdown
303 lines
10 KiB
Markdown
---
|
|
phase: 02-platform-skills
|
|
plan: 03
|
|
type: execute
|
|
wave: 2
|
|
depends_on: [02-01, 02-02]
|
|
files_modified:
|
|
- kb_pipeline/platforms/zhihu.py
|
|
- kb_pipeline/platforms/__init__.py
|
|
- kb_pipeline/__init__.py
|
|
- tests/test_platforms/test_zhihu.py
|
|
autonomous: true
|
|
requirements: [PLAT-03]
|
|
user_setup:
|
|
- service: zhihu
|
|
why: "知乎内容获取需要登录 Cookie"
|
|
env_vars:
|
|
- name: KB_ZHIHU_COOKIE
|
|
source: "浏览器开发者工具 -> Application -> Cookies -> zhihu.com -> 复制完整 Cookie 字符串"
|
|
dashboard_config:
|
|
- task: "获取知乎 Cookie"
|
|
location: "登录知乎后，在浏览器开发者工具中获取 Cookie"
|
|
|
|
must_haves:
|
|
truths:
|
|
- "输入知乎问答URL，获取问题标题、作者、回答内容"
|
|
- "输入知乎专栏URL，获取文章标题、作者、正文内容"
|
|
- "内容保存为Markdown文件（YAML Frontmatter + 正文）"
|
|
- "Cookie缺失时返回 AuthError 并提示配置"
|
|
- "检测到登录页面时返回 AuthError 并提示更新 Cookie"
|
|
artifacts:
|
|
- path: "kb_pipeline/platforms/zhihu.py"
|
|
provides: "知乎平台适配器"
|
|
exports: ["ZhihuAdapter"]
|
|
min_lines: 100
|
|
- path: "tests/test_platforms/test_zhihu.py"
|
|
provides: "知乎适配器测试"
|
|
min_lines: 60
|
|
key_links:
|
|
- from: "kb_pipeline/platforms/zhihu.py"
|
|
to: "kb_pipeline/config.py"
|
|
via: "get_settings().zhihu_cookie"
|
|
pattern: "get_settings().zhihu_cookie"
|
|
- from: "kb_pipeline/platforms/zhihu.py"
|
|
to: "kb-zhihu-metadata/zhihu_extractor.py"
|
|
via: "ZhihuPlaywrightExtractor class"
|
|
pattern: "from kb_zhihu_metadata"
|
|
- from: "kb_pipeline/platforms/zhihu.py"
|
|
to: "kb_pipeline/errors.py"
|
|
via: "AuthError for cookie issues"
|
|
pattern: "raise AuthError"
|
|
---
|
|
|
|
<objective>
|
|
创建知乎平台适配器，封装问答和专栏的内容获取逻辑，复用现有 ZhihuPlaywrightExtractor 实现。
|
|
|
|
Purpose: 为知乎问答和专栏提供统一的内容提取接口，支持 Cookie 认证和错误处理。
|
|
Output: ZhihuAdapter 实现类
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/02-platform-skills/02-CONTEXT.md
|
|
@.planning/phases/02-platform-skills/02-RESEARCH.md
|
|
|
|
<interfaces>
|
|
<!-- Phase 1 和 Plan 01/02 提供的接口 -->
|
|
|
|
From kb_pipeline/platforms/base.py (Plan 02):
|
|
```python
|
|
class PlatformAdapter(ABC):
|
|
@abstractmethod
|
|
def get_content_type(self, url: str) -> str: pass
|
|
|
|
@abstractmethod
|
|
async def fetch_metadata(self, content_id: str, content_type: str) -> Dict[str, Any]: pass
|
|
|
|
@abstractmethod
|
|
async def fetch_content(self, content_id: str, content_type: str) -> str: pass
|
|
|
|
@abstractmethod
|
|
def get_output_filename(self, content_id: str, content_type: str, suffix: str) -> str: pass
|
|
```
|
|
|
|
From kb_pipeline/config.py:
|
|
```python
|
|
class KBSettings(BaseSettings):
|
|
zhihu_cookie: SecretStr = Field(default=SecretStr(""), description="知乎 Cookie")
|
|
```
|
|
|
|
From kb_pipeline/errors.py:
|
|
```python
|
|
class AuthError(KBError):
|
|
def __init__(self, step_name, input_summary, auth_type=None, cause=None):
|
|
# suggestion: "检查 API Token 或 Cookie 是否有效"
|
|
```
|
|
|
|
From kb_pipeline/link_parser.py (Plan 01):
|
|
```python
|
|
@dataclass
|
|
class ParsedLink:
|
|
platform: str
|
|
content_type: str # "answer" | "zhuanlan"
|
|
content_id: str
|
|
```
|
|
</interfaces>
|
|
|
|
<existing_code>
|
|
<!-- 现有 ZhihuPlaywrightExtractor 的核心方法 -->
|
|
|
|
From kb-zhihu-metadata/zhihu_extractor.py:
|
|
```python
|
|
class ZhihuPlaywrightExtractor:
|
|
URL_PATTERNS = {
|
|
'zvideo': r'zhihu\.com/zvideo/(\d+)',
|
|
'zhuanlan': r'zhuanlan\.zhihu\.com/p/(\d+)',
|
|
'answer': r'zhihu\.com/question/\d+/answer/(\d+)',
|
|
'question': r'zhihu\.com/question/(\d+)',
|
|
}
|
|
|
|
def __init__(self, headless=True, timeout=30, cookie=None):
|
|
self.cookie = cookie
|
|
self.browser = PlaywrightBrowser(headless=headless, timeout=timeout)
|
|
|
|
def extract(self, url: str) -> Optional[Dict[str, Any]]:
|
|
# 返回 {type, title, author, content, content_markdown, publish_time, ...}
|
|
|
|
def _extract_answer_content(self) -> Dict[str, Any]: ...
|
|
def _extract_zhuanlan_content(self) -> Dict[str, Any]: ...
|
|
```
|
|
</existing_code>
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 1: Implement ZhihuAdapter for answer content</name>
|
|
<files>kb_pipeline/platforms/zhihu.py, tests/test_platforms/test_zhihu.py</files>
|
|
<behavior>
|
|
- Test 1: fetch_metadata("xxx", "answer") 返回正确的问答元数据
|
|
- Test 2: fetch_content("xxx", "answer") 返回 Markdown 格式的回答内容
|
|
- Test 3: Cookie 缺失时返回 AuthError
|
|
- Test 4: get_output_filename 返回正确格式 "zhihu_xxx_metadata.json"
|
|
</behavior>
|
|
<action>
|
|
创建 kb_pipeline/platforms/zhihu.py:
|
|
```python
|
|
from kb_pipeline.platforms.base import PlatformAdapter
|
|
from kb_pipeline.errors import AuthError, NetworkError
|
|
from kb_pipeline.config import get_settings
|
|
from kb_zhihu_metadata.zhihu_extractor import ZhihuPlaywrightExtractor
|
|
|
|
class ZhihuAdapter(PlatformAdapter):
|
|
PLATFORM = "zhihu"
|
|
|
|
def __init__(self, headless: bool = True, timeout: int = 30):
|
|
self.headless = headless
|
|
self.timeout = timeout
|
|
self._cookie = None
|
|
|
|
@property
|
|
def cookie(self) -> str:
|
|
if self._cookie is None:
|
|
settings = get_settings()
|
|
self._cookie = settings.zhihu_cookie.get_secret_value()
|
|
return self._cookie
|
|
|
|
def get_content_type(self, url: str) -> str:
|
|
# 从 URL 识别 answer 或 zhuanlan
|
|
if "answer" in url:
|
|
return "answer"
|
|
elif "zhuanlan" in url:
|
|
return "zhuanlan"
|
|
return "unknown"
|
|
|
|
async def fetch_metadata(self, content_id: str, content_type: str) -> Dict[str, Any]:
|
|
if not self.cookie:
|
|
raise AuthError(
|
|
step_name="zhihu_metadata",
|
|
input_summary=f"content_id={content_id}",
|
|
auth_type="zhihu_cookie",
|
|
)
|
|
# 使用 ZhihuPlaywrightExtractor 提取
|
|
pass
|
|
|
|
async def fetch_content(self, content_id: str, content_type: str) -> str:
|
|
# 返回 Markdown 格式内容
|
|
pass
|
|
|
|
def get_output_filename(self, content_id: str, content_type: str, suffix: str) -> str:
|
|
return f"zhihu_{content_id}_{suffix}"
|
|
```
|
|
|
|
创建 tests/test_platforms/test_zhihu.py:
|
|
- Mock ZhihuPlaywrightExtractor 的返回值
|
|
- 测试 Cookie 缺失时的 AuthError
|
|
- 测试问答内容提取
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_platforms/test_zhihu.py::test_answer_content -x -v</automated>
|
|
</verify>
|
|
<done>ZhihuAdapter 能正确提取知乎问答内容，Cookie 缺失时返回 AuthError</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 2: Implement ZhihuAdapter for zhuanlan content</name>
|
|
<files>kb_pipeline/platforms/zhihu.py, tests/test_platforms/test_zhihu.py</files>
|
|
<behavior>
|
|
- Test 1: fetch_metadata("xxx", "zhuanlan") 返回正确的专栏元数据
|
|
- Test 2: fetch_content("xxx", "zhuanlan") 返回 Markdown 格式的文章内容
|
|
- Test 3: 检测到登录页面时返回 AuthError 并提示更新 Cookie
|
|
</behavior>
|
|
<action>
|
|
扩展 kb_pipeline/platforms/zhihu.py:
|
|
1. 实现 zhuanlan 类型的 fetch_metadata 和 fetch_content
|
|
2. 添加登录页面检测逻辑:
|
|
```python
|
|
def _detect_login_page(self, result: Dict) -> bool:
|
|
"""检测是否被重定向到登录页面"""
|
|
if not result:
|
|
return False
|
|
title = result.get("title", "")
|
|
content = result.get("content", "")
|
|
return "登录" in title or len(content) < 100
|
|
```
|
|
3. 登录页面检测时抛出 AuthError:
|
|
```python
|
|
if self._detect_login_page(result):
|
|
raise AuthError(
|
|
step_name="zhihu_content",
|
|
input_summary=f"content_id={content_id}",
|
|
auth_type="zhihu_cookie",
|
|
cause=Exception("Cookie 可能已过期，请更新 KB_ZHIHU_COOKIE")
|
|
)
|
|
```
|
|
|
|
4. 扩展测试覆盖专栏场景
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/test_platforms/test_zhihu.py::test_zhuanlan_content -x -v</automated>
|
|
</verify>
|
|
<done>ZhihuAdapter 能正确提取知乎专栏内容，检测登录页面时返回 AuthError</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 3: Update package exports</name>
|
|
<files>kb_pipeline/__init__.py, kb_pipeline/platforms/__init__.py</files>
|
|
<action>
|
|
1. 更新 kb_pipeline/platforms/__init__.py:
|
|
```python
|
|
from .base import PlatformAdapter
|
|
from .bilibili import BilibiliAdapter
|
|
from .zhihu import ZhihuAdapter
|
|
|
|
__all__ = ["PlatformAdapter", "BilibiliAdapter", "ZhihuAdapter"]
|
|
```
|
|
|
|
2. 更新 kb_pipeline/__init__.py:
|
|
```python
|
|
from .platforms import PlatformAdapter, BilibiliAdapter, ZhihuAdapter
|
|
|
|
__all__ = [
|
|
# ... existing exports ...
|
|
"PlatformAdapter",
|
|
"BilibiliAdapter",
|
|
"ZhihuAdapter",
|
|
]
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>python -c "from kb_pipeline import ZhihuAdapter; print('OK')"</automated>
|
|
</verify>
|
|
<done>ZhihuAdapter 可从 kb_pipeline 包根导入</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
- 所有测试通过: `pytest tests/test_platforms/test_zhihu.py -v`
|
|
- 导入测试: `python -c "from kb_pipeline import ZhihuAdapter"`
|
|
- 覆盖率: `pytest tests/test_platforms/ --cov=kb_pipeline/platforms --cov-report=term-missing`
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- [ ] ZhihuAdapter 正确提取知乎问答内容
|
|
- [ ] ZhihuAdapter 正确提取知乎专栏内容
|
|
- [ ] Cookie 缺失时返回 AuthError
|
|
- [ ] 检测到登录页面时返回 AuthError 并提示更新 Cookie
|
|
- [ ] 内容保存为 Markdown 格式，包含 YAML Frontmatter
|
|
- [ ] 所有测试通过（至少 6 个测试用例）
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/02-platform-skills/02-03-SUMMARY.md`
|
|
</output>
|