starmen1265/kb-skills
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(09): complete phase 9 planning with 2 plans in 2 waves

Browse Source
This commit is contained in:
starmen1265
2026-03-17 22:54:48 +08:00
parent 1c42443dce
commit bee696b96d
7 changed files with 681 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/phases/07-login-auth/07-VERIFICATION.md
+151
View File
@@ -0,0 +1,151 @@
---
phase: 07-login-auth
verified: 2026-03-17T12:30:00Z
status: passed
score: 12/12 must-haves verified
re_verification: No
human_verification:
- test: "运行 python3 kb-workflow/kb_workflow.py login 进行实际二维码扫码登录"
expected: "二维码图片生成在中间目录，扫码后 Cookie 保存到 ~/.kb_env"
why_human: "需要真实的知乎 APP 扫码，无法自动化测试"
- test: "运行 python3 kb-workflow/kb_workflow.py login --status 检查 Cookie 状态"
expected: "返回用户名和用户 ID 或提示 Cookie 过期"
why_human: "需要真实有效的 Cookie 才能验证 API 返回"
- test: "运行 python3 kb-workflow/kb_workflow.py login --import-cookie cookies.json 导入 Cookie"
expected: "Cookie 从 JSON 文件正确导入并保存"
why_human: "需要真实的 Chrome Cookie Editor 导出文件"
---
# Phase 7: 登录认证体系 Verification Report
**Phase Goal:** 建立知乎登录认证机制，支持二维码扫码和手动 Cookie 两种方式，实现 Cookie 持久化和过期检测
**Verified:** 2026-03-17T12:30:00Z
**Status:** passed
**Re-verification:** No - initial verification
## Goal Achievement
### Observable Truths
| # | Truth | Status | Evidence |
| --- | --- | --- | --- |
| 1 | 用户可以通过命令行显示二维码并扫码登录知乎 | VERIFIED | ZhihuLogin.login_by_qrcode() 方法实现完整 (276 行)，login CLI 子命令存在 |
| 2 | 扫码失败时用户可以手动设置 Cookie 进行认证 | VERIFIED | parse_json_cookies() 支持 Chrome Cookie Editor JSON 格式，--import-cookie 参数实现 |
| 3 | Cookie 自动保存到本地文件，下次运行自动复用 | VERIFIED | CookieManager.save_cookie() 保存到 ~/.kb_env，get_settings() 自动加载 |
| 4 | Cookie 过期时系统自动检测并提示用户重新登录 | VERIFIED | validate_cookie() 通过 /api/v4/me 检测，返回 CookieValidationResult |
| 5 | 用户可以访问知乎登录页面 | VERIFIED | LOGIN_URL = "https://www.zhihu.com/signin"，goto() 调用正确 |
| 6 | 系统可以从 Canvas 元素截取二维码图片 | VERIFIED | _find_qrcode_canvas() 多选择器回退，canvas.screenshot() 调用 |
| 7 | 二维码图片保存到中间目录 | VERIFIED | _save_qrcode_image() 保存为 zhihu_qrcode_{timestamp}.png |
| 8 | 系统可以轮询检测 z_c0 Cookie 判断登录状态 | VERIFIED | check_login_cookie() + tenacity Retrying 实现 60 秒轮询 |
| 9 | 用户可以从 JSON 文件导入 Cookie | VERIFIED | parse_json_cookies() 实现并测试通过 |
| 10 | Cookie 可以保存到 ~/.kb_env 文件 | VERIFIED | CookieManager.save_cookie() 写入文件，权限 600 |
| 11 | 保存的 Cookie 下次运行时可以自动加载 | VERIFIED | load_cookie() 读取，get_settings.cache_clear() 确保刷新 |
| 12 | 用户可以查看当前 Cookie 状态 | VERIFIED | login --status 参数调用 validate_cookie() |
**Score:** 12/12 truths verified
### Required Artifacts
| Artifact | Expected | Status | Details |
| --- | --- | --- | --- |
| `kb_workflow/auth/zhihu_login.py` | 二维码登录核心逻辑 (min 100 lines) | VERIFIED | 276 行，导出 ZhihuLogin, ZhihuLoginResult, CookieValidationResult |
| `kb_workflow/auth/cookie_manager.py` | Cookie 导入和持久化 (min 80 lines) | VERIFIED | 120 行，导出 CookieManager, parse_json_cookies |
| `kb_workflow/kb_workflow.py` | login CLI 子命令 | VERIFIED | login_parser subparser，login_command() 函数 |
| `tests/test_auth/test_qrcode_login.py` | 登录逻辑单元测试 | VERIFIED | 8 个测试全部通过 |
| `tests/test_auth/test_cookie_import.py` | Cookie 导入测试 | VERIFIED | 6 个测试全部通过 |
| `tests/test_auth/test_cookie_persistence.py` | Cookie 持久化测试 | VERIFIED | 8 个测试全部通过 |
| `tests/test_auth/test_cookie_validation.py` | Cookie 验证测试 | VERIFIED | 5 个测试全部通过 |
| `tests/test_auth/test_cli_login.py` | CLI 命令测试 | VERIFIED | 6 个测试全部通过 |
### Key Link Verification
| From | To | Via | Status | Details |
| --- | --- | --- | --- | --- |
| `zhihu_login.py` | `PlaywrightBrowser` | import | WIRED | `from kb_workflow.metadata.zhihu_extractor import PlaywrightBrowser` |
| `ZhihuLogin.login_by_qrcode()` | `canvas.screenshot()` | Playwright API | WIRED | canvas.screenshot() 调用存在 |
| `CookieManager.save_cookie()` | `~/.kb_env` | 文件写入 | WIRED | Path.home() / ".kb_env"，chmod 600 |
| `parse_json_cookies()` | Chrome Cookie Editor JSON | json.load + 过滤 | WIRED | zhihu.com 域名过滤实现 |
| `kb_workflow.py login` | `ZhihuLogin.login_by_qrcode()` | 同步调用 | WIRED | login.login_by_qrcode() 调用 |
| `ZhihuLogin.login_by_qrcode()` | `CookieManager.save_cookie()` | 登录成功后保存 | WIRED | manager.save_cookie(result.cookie_str) |
| `validate_cookie()` | `/api/v4/me` | Playwright request API | WIRED | VALIDATION_URL = "https://www.zhihu.com/api/v4/me" |
### Requirements Coverage
| Requirement | Source Plan | Description | Status | Evidence |
| --- | --- | --- | --- | --- |
| AUTH-01 | 07-01 | 用户可通过二维码扫码登录知乎 | SATISFIED | ZhihuLogin.login_by_qrcode() 完整实现，测试通过 |
| AUTH-02 | 07-02 | 用户可手动设置 Cookie 进行认证 | SATISFIED | parse_json_cookies() + --import-cookie 参数 |
| AUTH-03 | 07-02 | Cookie 自动持久化到本地文件，下次自动复用 | SATISFIED | CookieManager.save_cookie/load_cookie + get_settings 缓存清除 |
| AUTH-04 | 07-03 | Cookie 过期时自动检测并提示重新登录 | SATISFIED | validate_cookie() + CookieValidationResult |
### Anti-Patterns Found
| File | Line | Pattern | Severity | Impact |
| --- | --- | --- | --- | --- |
| (none) | - | - | - | auth 模块无 TODO/FIXME/placeholder |
**Note:** 在 `kb_workflow/pipeline/platforms/bilibili.py` 中存在 TODO 注释，但这不在阶段7范围内。
### Human Verification Required
虽然所有自动化测试通过，以下场景需要人工验证：
#### 1. 二维码扫码登录端到端测试
**Test:** 运行 `python3 kb-workflow/kb_workflow.py login`
**Expected:**
- 二维码图片生成在 `~/work/intermediate/zhihu_qrcode_*.png`
- 使用知乎 APP 扫码后命令返回 `{"success": true, "cookie_str": "..."}`
- `~/.kb_env` 文件包含 `ZHIHU_COOKIE="..."` 行
**Why human:** 需要真实的知乎 APP 扫码，无法自动化测试
#### 2. Cookie 状态检查
**Test:** 运行 `python3 kb-workflow/kb_workflow.py login --status`
**Expected:**
- 有效 Cookie: 返回 `{"valid": true, "username": "...", "user_id": "..."}`
- 过期 Cookie: 返回 `{"valid": false, "reason": "Cookie expired"}`
**Why human:** 需要真实有效的 Cookie 才能验证 API 返回
#### 3. JSON Cookie 导入
**Test:** 使用 Chrome Cookie Editor 导出 cookies.json，运行 `python3 kb-workflow/kb_workflow.py login --import-cookie cookies.json`
**Expected:** Cookie 正确导入并保存到 ~/.kb_env
**Why human:** 需要真实的 Chrome Cookie Editor 导出文件
### Test Results
```
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.0.2, pluggy-1.6.0
tests/test_auth/ - 33 items
PASSED: 33/33 tests
- test_qrcode_login.py: 8 tests
- test_cookie_import.py: 6 tests
- test_cookie_persistence.py: 8 tests
- test_cookie_validation.py: 5 tests
- test_cli_login.py: 6 tests
```
### Summary
**Phase 7 (登录认证体系) 验证通过:**
1. **AUTH-01 二维码扫码登录** - ZhihuLogin 类实现完整，支持多选择器回退、可配置超时、二维码保存
2. **AUTH-02 手动 Cookie 导入** - parse_json_cookies 支持 Chrome Cookie Editor 格式，域名过滤
3. **AUTH-03 Cookie 持久化** - CookieManager 保存到 ~/.kb_env，权限 600，缓存清除
4. **AUTH-04 Cookie 过期检测** - validate_cookie 通过 /api/v4/me 验证，返回详细状态
所有 33 个单元测试通过，关键连接点验证通过，无阻塞性 anti-pattern。
**Ready for Phase 8 (内容爬取实现)**
---
_Verified: 2026-03-17T12:30:00Z_
_Verifier: Claude (gsd-verifier)_
.planning/phases/09-pipeline-integration/09-02-plan.md
+209
View File
@@ -0,0 +1,209 @@
---
phase: 09-pipeline-integration
plan: 02
type: execute
wave: 1
depends_on: [09-01]
files_modified: []
autonomous: true
requirements: []
user_setup: []
must_haves:
truths:
- "用户可通过命令行测试完整 Pipeline 流程"
- "Pipeline 正确处理知乎链接并输出内容"
artifacts:
- path: "tests/test_platforms/test_zhihu.py"
provides: "测试验证 ZhihuAdapter 整合"
min_lines: 100
exports: ["test_cookie_expired_triggers_login", "test_comment_format", "test_crawl_with_retry"]
- path: "kb_workflow/pipeline/platforms/zhihu.py"
provides: "ZhihuAdapter implementation"
min_lines: 50
- path: "tests/test_processors/test_download.py"
provides: "Download processor integration test"
key_links:
- from: "ZhihuAdapter" to: "DownloadProcessor"
via: "platform = adapter mapping"
pattern: "platform_adapter_map"
- from: "DownloadProcessor" to: "content files"
via: "content_dir / filename"
---
<objective>
将 ZhihuAdapter 从使用 ZhihuPlaywrightExtractor (DOM 解析) 切换到使用 ZhihuCrawler (API 拦截)。
目的: 提爬取可靠性和，支持评论,保持 Pipeline 其容性
输出: 更新后的 ZhihuAdapter, 更新后的测试用例
</objective>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/08-crawler/08-03-SUMMARY.md
@.planning/phases/09-pipeline-integration/09-RESEARCH.md
@.planning/phases/09-pipeline-integration/09-CONTEXT.md
@.planning/phases/09-pipeline-integration/09-VALIDation.md
</context>
<execution_context>
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
</execution_context>
<interfaces>
<!-- Key types and contracts the executor needs. Executor should use these directly - no codebase exploration needed. -->
from kb_workflow/pipeline/platforms/base.py:
```python
class PlatformAdapter(ABC):
@abstractmethod
def get_content_type(self, url: str) -> str:
pass
@abstractmethod
async def fetch_metadata(self, content_id: str, content_type: str, source_url: str = None) -> Dict[str, Any]:
pass
@abstractmethod
async def fetch_content(self, content_id: str, content_type: str, source_url: str = None) -> str:
pass
@abstractmethod
def get_output_filename(self, content_id: str, content_type: str, suffix: str) -> str:
pass
```
From kb_workflow/crawler/models.py
```python
class CrawlResult(BaseModel):
success: bool
content_type: str
data: Optional[Union[AnswerData, ArticleData]]
comments: Optional[List[CommentData]]
error: Optional[str]
class AnswerData(BaseModel):
id: int
question_id: int
question_title: str
content: str
excerpt: str
metadata: ContentMetadata
class ArticleData(BaseModel):
id: int
title: str
content: str
excerpt: str
metadata: ContentMetadata
class CommentData(BaseModel):
id: int
content: str
author: Optional[CommentAuthor]
created_time: Optional[int]
like_count: int
child_count: int
is_author: bool
class CommentsResult(BaseModel):
content_type: str
content_id: int
comments: List[CommentData]
total_count: int
class CommentAuthor(BaseModel):
name: str
url_token: Optional[str]
class ContentMetadata(BaseModel):
author: Optional[AuthorInfo]
voteup_count: int
comment_count: int
created_time: Optional[int]
updated_time: Optional[int]
class AuthorInfo(BaseModel):
name: str
url_token: Optional[str]
headline: Optional[str]
`` </interfaces>
from kb_workflow/crawler/zhihu_crawler.py
```python
class ZhihuCrawler:
def crawl_answer(self, url: str, include_comments: bool = False) -> CrawlResult:
...
def crawl_article(self, url: str) -> CrawlResult:
...
def crawl_comments(self, content_type: str, content_id: int, include_comments: bool = True) -> CommentsResult:
...
```
from kb_workflow/auth/zhihu_login.py
```python
class ZhihuLogin
def login_by_qrcode(self) -> ZhihuLoginResult:
...
def validate_cookie(self, cookie_str: str = None) -> CookieValidationResult
"""
</interfaces>
<tasks>
<task type="auto">
<name>Task 1: Add Cookie expiration auto login tests</name>
<files>tests/test_platforms/test_zhihu.py</files>
<action>
添加 test用例验证 Cookie 过期时自动触发扫码登录流程:
使用 pytest fixtures mock ZhihuLogin 和 ZhihuCrawler:
测试场景:
1. Cookie 有效时,爬取正常返回数据
2. Cookie 过期时,触发扫码登录流程
3. 触发登录后重试成功
4. 触发登录失败时,抛出异常
验证正确处理各种情况
</action>
<verify>
<automated>pytest tests/test_platforms/test_zhihu.py -k "test_cookie_expired" -v</automated>
</verify>
<done>
所有 Cookie 过期自动登录测试用例通过
</done>
</task>
<task type="auto">
<name>Task 2: Add comment formatting tests</name>
<files>tests/test_platforms/test_zhihu.py</files>
<action>
添加测试用例验证评论格式化是否正确追加到正文末尾
使用 mock 数据验证格式正确
测试场景:
1. 有评论时,格式化为包含评论列表的 Markdown
2. 无评论时,返回空字符串
</action>
<verify>
<automated>pytest tests/test_platforms/test_zhihu.py -k "test_comment_format" -v</automated>
</verify>
<done>
所有评论格式化测试用例通过
</done>
</task>
</tasks>
<verification>
1. 运行单元测试: `pytest tests/test_platforms/test_zhihu.py -x -v`
2. 运行下载处理器集成测试: `pytest tests/test_processors/test_download.py -k "zhihu" -v`
</verification>
<success_criteria>
- 所有单元测试通过
- 下载处理器集成测试通过
- ZhihuAdapter 与 ZhihuCrawler 正确协作
</success_criteria>
<output>
After completion, create `.planning/phases/09-pipeline-integration/09-02-SUMMARY.md`
</output>
.planning/phases/09-pipeline-integration/09-VALIDATION.md
+81
View File
@@ -0,0 +1,81 @@
---
phase: 09
slug: pipeline-integration
status: draft
nyquist_compliant: false
wave_0_complete: false
created: 2026-03-17
---
# Phase 09 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | pytest |
| **Config file** | pytest.ini |
| **Quick run command** | `pytest tests/test_platforms/test_zhihu.py -x -v` |
| **Full suite command** | `pytest tests/ -v` |
| **Estimated runtime** | ~30 seconds |
---
## Sampling Rate
- **After every task commit:** Run `pytest tests/test_platforms/test_zhihu.py -x -v`
- **After every plan wave:** Run `pytest tests/test_platforms/ tests/test_processors/test_download.py -v`
- **Before `/gsd:verify-work`:** Full suite must be green
- **Max feedback latency:** 30 seconds
---
## Per-Task Verification Map
| Task ID | Plan | Wave | Requirement | Test Type | Automated Command | File Exists | Status |
|---------|------|------|-------------|-----------|-------------------|-------------|--------|
| 09-01-01 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_fetch" -v` | ✅ 需修改 | ⬜ pending |
| 09-01-02 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_crawl" -v` | ✅ 需修改 | ⬜ pending |
| 09-01-03 | 01 | 1 | PIPE-02 | integration | `pytest tests/test_processors/test_download.py -k "zhihu" -v` | ✅ 已存在 | ⬜ pending |
| 09-01-04 | 01 | 1 | PIPE-03 | unit | `pytest tests/test_pipeline/test_idempotency.py -v` | ✅ 已存在 | ⬜ pending |
| 09-01-05 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_cookie" -v` | ❌ W0 | ⬜ pending |
| 09-01-06 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_comment" -v` | ❌ W0 | ⬜ pending |
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
---
## Wave 0 Requirements
- [ ] `tests/test_platforms/test_zhihu.py` — 更新测试使用 ZhihuCrawler mock
- [ ] `tests/test_platforms/test_zhihu.py::test_cookie_expired_login` — Cookie 过期自动触发登录流程测试
- [ ] `tests/test_platforms/test_zhihu.py::test_comment_format` — 评论追加到正文格式验证测试
- [ ] `tests/test_platforms/test_zhihu.py::test_crawlresult_to_metadata` — CrawlResult 到 metadata 转换正确性测试
*现有测试基础设施完善，主要是更新和新增 ZhihuAdapter 相关测试*
---
## Manual-Only Verifications
| Behavior | Requirement | Why Manual | Test Instructions |
|----------|-------------|------------|-------------------|
| 扫码登录实际流程 | PIPE-01 | 需要 Zhihu API 真实交互 | 1. 删除 Cookie 文件 2. 运行 Pipeline 3. 验证二维码显示 4. 扫码后流程继续 |
| 评论实际爬取 | PIPE-01 | 需要 Zhihu API 真实交互 | 1. 运行 Pipeline 处理知乎链接 2. 验证评论追加到 content.md |
---
## Validation Sign-Off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] Wave 0 covers all MISSING references
- [ ] No watch-mode flags
- [ ] Feedback latency < 30s
- [ ] `nyquist_compliant: true` set in frontmatter
**Approval:** pending
.planning/phases/09-pipeline-integration/09-VALIDation.md
+66
View File
@@ -0,0 +1,66 @@
---
phase: 09
slug: pipeline-integration
status: draft
nyquist_compliant: false
wave_0_complete: false
created: 2026-03-17
---
# Phase 09 — Validation Strategy
> Per-phase validation contract for feedback sampling during execution.
---
## Test Infrastructure
| Property | Value |
|----------|-------|
| **Framework** | pytest |
| **Config file** | pytest.ini |
| **Quick run command** | `pytest tests/test_platforms/test_zhihu.py -x -v` |
| **full suite command** | `pytest tests/ -v` |
| **estimated runtime** | ~30 seconds |
---
## Sampling Rate
- **After every task commit:** Run `pytest tests/test_platforms/test_zhihu.py -x -v`
- **after every plan wave:** run `pytest tests/test_platforms/ tests/test_processors/test_download.py -v`
- **Before `/gsd:verify-work`:** Full suite must be green
- **Max feedback latency:** 30 seconds
---
## Per-task verification map
| Task ID | Plan | Wave | Requirement | Test Type | Automated command | File exists | status |
|---------|------|------|-------------|-----------|-------------------|-------------|--------|
| 09-01-01 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_fetch" -v` | ✅ needs update | ⬜ pending |
| 09-01-02 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_crawl" -v` | ✅ needs update | ⬜ pending |
| 09-01-03 | 01 | 1 | PIPE-02 | integration | `pytest tests/test_processors/test_download.py -k "zhihu" -v` | ✅ exists | ⬜ pending |
| 09-01-04 | 01 | 1 | PIPE-03 | unit | `pytest tests/test_pipeline/test_idempotency.py -v` | ✅ exists | ⬜ pending |
| 09-01-05 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_cookie" -v` | ❌ W0 | ⬜ pending |
| 09-01-06 | 01 | 1 | PIPE-01 | unit | `pytest tests/test_platforms/test_zhihu.py -k "test_comment" -v` | ❌ W0 | ⬜ pending |
*Status: ⬜ pending · ✅ green · ❌ red | ⚠️ flaky*
---
## Wave 0 requirements
- [ ] `tests/test_platforms/test_zhihu.py` - 更新测试使用 ZhihuCrawler mock
- [ ] `tests/test_platforms/test_zhihu.py::test_cookie_expired_triggers_login` - Cookie 过期自动触发登录流程测试
- [ ] `tests/test_platforms/test_zhihu.py::test_comment_format` - 评论追加到正文格式验证测试
- [ ] `tests/test_platforms/test_zhihu.py::test_crawlresult_to_metadata` - CrawlResult 到 metadata 转换正确性测试
*现有测试基础设施完善,主要是更新和新增 ZhihuAdapter 相关测试
---
## manual-only verifications
| behavior | requirement | why manual | test instructions |
|----------|-------------|------------|-------------------|
| 扫码登录实际流程 | PIPE-01 | 需要 Zhihu API 真实交互 | 1. 删除 Cookie 文件 2. 运行 Pipeline 3. 陑证二维码显示 4. 扫码后流程继续 |
| 评论实际爬取 | PIPE-01 | 需要 Zhihu API 瓟实交互 | 1. 运行 Pipeline 处理知乎链接 2. 验证评论追加到 content.md |
---
## validation sign-off
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
- [ ] wave 0 covers all MISSING references
- [ ] no watch-mode flags
- [ ] feedback latency < 30s
- [ ] `nyquist_compliant: true` set in frontmatter
**approval:** pending