docs: create milestone v1.2 roadmap (3 phases)
This commit is contained in:
+17
-17
@@ -111,25 +111,25 @@
|
|||||||
|
|
||||||
| Requirement | Phase | Status |
|
| Requirement | Phase | Status |
|
||||||
|-------------|-------|--------|
|
|-------------|-------|--------|
|
||||||
| AUTH-01 | — | Pending |
|
| AUTH-01 | Phase 7 | Pending |
|
||||||
| AUTH-02 | — | Pending |
|
| AUTH-02 | Phase 7 | Pending |
|
||||||
| AUTH-03 | — | Pending |
|
| AUTH-03 | Phase 7 | Pending |
|
||||||
| AUTH-04 | — | Pending |
|
| AUTH-04 | Phase 7 | Pending |
|
||||||
| CONT-01 | — | Pending |
|
| CONT-01 | Phase 8 | Pending |
|
||||||
| CONT-02 | — | Pending |
|
| CONT-02 | Phase 8 | Pending |
|
||||||
| CONT-03 | — | Pending |
|
| CONT-03 | Phase 8 | Pending |
|
||||||
| CONT-04 | — | Pending |
|
| CONT-04 | Phase 8 | Pending |
|
||||||
| NET-01 | — | Pending |
|
| NET-01 | Phase 8 | Pending |
|
||||||
| NET-02 | — | Pending |
|
| NET-02 | Phase 8 | Pending |
|
||||||
| NET-03 | — | Pending |
|
| NET-03 | Phase 8 | Pending |
|
||||||
| PIPE-01 | — | Pending |
|
| PIPE-01 | Phase 9 | Pending |
|
||||||
| PIPE-02 | — | Pending |
|
| PIPE-02 | Phase 9 | Pending |
|
||||||
| PIPE-03 | — | Pending |
|
| PIPE-03 | Phase 9 | Pending |
|
||||||
|
|
||||||
**v1.2 Coverage:**
|
**v1.2 Coverage:**
|
||||||
- Requirements: 14 total
|
- Requirements: 14 total
|
||||||
- Mapped to phases: 0
|
- Mapped to phases: 14
|
||||||
- Unmapped: 14 ⚠️
|
- Unmapped: 0 ✓
|
||||||
|
|
||||||
### v1.1 Traceability (Complete)
|
### v1.1 Traceability (Complete)
|
||||||
|
|
||||||
@@ -158,4 +158,4 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
*Requirements defined: 2026-03-17*
|
*Requirements defined: 2026-03-17*
|
||||||
*Last updated: 2026-03-17 after v1.2 milestone started*
|
*Last updated: 2026-03-17 after v1.2 roadmap created*
|
||||||
|
|||||||
+64
-19
@@ -3,7 +3,8 @@
|
|||||||
## Milestones
|
## Milestones
|
||||||
|
|
||||||
- ✅ **v1.0 Pipeline Architecture** - Phases 1-4 (shipped 2026-03-15)
|
- ✅ **v1.0 Pipeline Architecture** - Phases 1-4 (shipped 2026-03-15)
|
||||||
- 🚧 **v1.1 Architecture Refactor** - Phases 5-6 (in progress)
|
- ✅ **v1.1 Architecture Refactor** - Phases 5-6 (shipped 2026-03-16)
|
||||||
|
- 🚧 **v1.2 Zhihu Crawler** - Phases 7-9 (in progress)
|
||||||
|
|
||||||
## Phases
|
## Phases
|
||||||
|
|
||||||
@@ -46,11 +47,10 @@ Plans:
|
|||||||
|
|
||||||
</details>
|
</details>
|
||||||
|
|
||||||
### 🚧 v1.1 Architecture Refactor (In Progress)
|
<details>
|
||||||
|
<summary>✅ v1.1 Architecture Refactor (Phases 5-6) - SHIPPED 2026-03-16</summary>
|
||||||
|
|
||||||
**Milestone Goal:** 简化代码结构和优化文件组织 — 统一中间目录、添加归档步骤、整合元数据技能和 Pipeline 模块
|
### Phase 5: 归档与目录优化
|
||||||
|
|
||||||
#### Phase 5: 归档与目录优化
|
|
||||||
**Goal**: 统一中间目录结构,添加自动归档步骤,优化最终目录布局
|
**Goal**: 统一中间目录结构,添加自动归档步骤,优化最终目录布局
|
||||||
**Depends on**: Phase 4
|
**Depends on**: Phase 4
|
||||||
**Requirements**: INT-01, INT-02, INT-03, ARC-01, ARC-02, ARC-03, ARC-04, OUT-01, OUT-02, OUT-03, OUT-04
|
**Requirements**: INT-01, INT-02, INT-03, ARC-01, ARC-02, ARC-03, ARC-04, OUT-01, OUT-02, OUT-03, OUT-04
|
||||||
@@ -60,15 +60,10 @@ Plans:
|
|||||||
3. 最终目录结构为 `{KB_ARCHIVE_PATH}/{platform}/{YYYYMMDD}_{content_id}_analysis.md`
|
3. 最终目录结构为 `{KB_ARCHIVE_PATH}/{platform}/{YYYYMMDD}_{content_id}_analysis.md`
|
||||||
4. 归档时检测重复文件,询问用户是否覆盖
|
4. 归档时检测重复文件,询问用户是否覆盖
|
||||||
5. 不自动清理中间文件(用户手动维护)
|
5. 不自动清理中间文件(用户手动维护)
|
||||||
**Plans**: 3 plans
|
**Plans**: 3 plans (all complete)
|
||||||
|
|
||||||
Plans:
|
### Phase 6: 代码整合
|
||||||
- [x] 05-01-PLAN.md — 中间目录统一和配置 (INT-01, INT-02, INT-03)
|
**Goal**: 将独立的元数据技能和 Pipeline 模块整合进 kb-workflow
|
||||||
- [x] 05-02-PLAN.md — ArchiveProcessor 归档处理器 (ARC-01, ARC-02, ARC-03, ARC-04)
|
|
||||||
- [x] 05-03-PLAN.md — 最终目录结构验证和测试 (OUT-01, OUT-02, OUT-03, OUT-04)
|
|
||||||
|
|
||||||
#### Phase 6: 代码整合
|
|
||||||
**Goal**: 将独立的元数据技能和 Pipeline 模块整合进 kb-workflow,消除 sys.path 操作,建立标准 Python 包结构
|
|
||||||
**Depends on**: Phase 5
|
**Depends on**: Phase 5
|
||||||
**Requirements**: MET-01, MET-02, MET-03, MET-04, MET-05, PIP-01, PIP-02, PIP-03, PIP-04
|
**Requirements**: MET-01, MET-02, MET-03, MET-04, MET-05, PIP-01, PIP-02, PIP-03, PIP-04
|
||||||
**Success Criteria** (what must be TRUE):
|
**Success Criteria** (what must be TRUE):
|
||||||
@@ -78,17 +73,64 @@ Plans:
|
|||||||
4. 原独立的 kb-bilibili-metadata、kb-zhihu-metadata、kb_pipeline 目录已删除
|
4. 原独立的 kb-bilibili-metadata、kb-zhihu-metadata、kb_pipeline 目录已删除
|
||||||
5. 所有 sys.path 动态操作已删除或重构
|
5. 所有 sys.path 动态操作已删除或重构
|
||||||
6. Pipeline 类可从 kb_workflow.pipeline 包级别导入
|
6. Pipeline 类可从 kb_workflow.pipeline 包级别导入
|
||||||
**Plans**: 3 plans
|
**Plans**: 3 plans (all complete)
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
|
### 🚧 v1.2 Zhihu Crawler (In Progress)
|
||||||
|
|
||||||
|
**Milestone Goal:** 构建可靠的知乎内容爬取体系 — 解决 Cookie 过期问题,支持自动登录维持
|
||||||
|
|
||||||
|
#### Phase 7: 登录认证体系
|
||||||
|
**Goal**: 建立知乎登录认证机制,支持二维码扫码和手动 Cookie 两种方式,实现 Cookie 持久化和过期检测
|
||||||
|
**Depends on**: Phase 6
|
||||||
|
**Requirements**: AUTH-01, AUTH-02, AUTH-03, AUTH-04
|
||||||
|
**Success Criteria** (what must be TRUE):
|
||||||
|
1. 用户可以通过命令行显示二维码并扫码登录知乎
|
||||||
|
2. 扫码失败时用户可以手动设置 Cookie 进行认证
|
||||||
|
3. Cookie 自动保存到本地文件,下次运行自动复用
|
||||||
|
4. Cookie 过期时系统自动检测并提示用户重新登录
|
||||||
|
**Plans**: TBD
|
||||||
|
|
||||||
Plans:
|
Plans:
|
||||||
- [x] 06-01-PLAN.md — 元数据技能整合 (MET-01, MET-02, MET-03, MET-04, MET-05)
|
- [ ] 07-01: 二维码扫码登录实现 (AUTH-01)
|
||||||
- [x] 06-02-PLAN.md — Pipeline 模块整合 (PIP-01, PIP-02, PIP-03, PIP-04)
|
- [ ] 07-02: 手动 Cookie 设置和持久化 (AUTH-02, AUTH-03)
|
||||||
- [ ] 06-03-PLAN.md — Gap Closure: 消除 sys.path 操作和补充 Pipeline 导出 (MET-05, PIP-04)
|
- [ ] 07-03: Cookie 过期检测机制 (AUTH-04)
|
||||||
|
|
||||||
|
#### Phase 8: 内容爬取实现
|
||||||
|
**Goal**: 实现知乎问答、专栏/文章、评论的完整爬取功能,包含网络反爬措施
|
||||||
|
**Depends on**: Phase 7
|
||||||
|
**Requirements**: CONT-01, CONT-02, CONT-03, CONT-04, NET-01, NET-02, NET-03
|
||||||
|
**Success Criteria** (what must be TRUE):
|
||||||
|
1. 用户可以爬取知乎问答(问题标题、描述、回答内容、元数据)
|
||||||
|
2. 用户可以爬取知乎专栏/文章(标题、正文、元数据)
|
||||||
|
3. 用户可以选择是否爬取内容下的评论
|
||||||
|
4. 爬取结果包含作者信息、发布时间、点赞数等完整元数据
|
||||||
|
5. 用户可以配置代理 IP 以降低被封风险
|
||||||
|
**Plans**: TBD
|
||||||
|
|
||||||
|
Plans:
|
||||||
|
- [ ] 08-01: 问答和专栏内容爬取 (CONT-01, CONT-02, CONT-04)
|
||||||
|
- [ ] 08-02: 评论爬取功能 (CONT-03)
|
||||||
|
- [ ] 08-03: 网络反爬措施 (NET-01, NET-02, NET-03)
|
||||||
|
|
||||||
|
#### Phase 9: Pipeline 整合
|
||||||
|
**Goal**: 将新爬虫实现整合进现有 Pipeline 架构,替换原有知乎处理器
|
||||||
|
**Depends on**: Phase 8
|
||||||
|
**Requirements**: PIPE-01, PIPE-02, PIPE-03
|
||||||
|
**Success Criteria** (what must be TRUE):
|
||||||
|
1. 知乎链接使用新爬虫处理器,无需手动设置 Cookie
|
||||||
|
2. 新处理器与现有 Pipeline 架构完全兼容
|
||||||
|
3. 支持断点续传和幂等执行(与其他处理器行为一致)
|
||||||
|
**Plans**: TBD
|
||||||
|
|
||||||
|
Plans:
|
||||||
|
- [ ] 09-01: ZhihuProcessor 替换和整合 (PIPE-01, PIPE-02, PIPE-03)
|
||||||
|
|
||||||
## Progress
|
## Progress
|
||||||
|
|
||||||
**Execution Order:**
|
**Execution Order:**
|
||||||
Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6
|
Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9
|
||||||
|
|
||||||
| Phase | Milestone | Plans Complete | Status | Completed |
|
| Phase | Milestone | Plans Complete | Status | Completed |
|
||||||
|-------|-----------|----------------|--------|-----------|
|
|-------|-----------|----------------|--------|-----------|
|
||||||
@@ -97,4 +139,7 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6
|
|||||||
| 3. Processors | v1.0 | 3/3 | Complete | 2026-03-15 |
|
| 3. Processors | v1.0 | 3/3 | Complete | 2026-03-15 |
|
||||||
| 4. Pipeline | v1.0 | 2/2 | Complete | 2026-03-15 |
|
| 4. Pipeline | v1.0 | 2/2 | Complete | 2026-03-15 |
|
||||||
| 5. 归档与目录优化 | v1.1 | 3/3 | Complete | 2026-03-16 |
|
| 5. 归档与目录优化 | v1.1 | 3/3 | Complete | 2026-03-16 |
|
||||||
| 6. 代码整合 | v1.1 | 2/3 | In Progress | - |
|
| 6. 代码整合 | v1.1 | 3/3 | Complete | 2026-03-16 |
|
||||||
|
| 7. 登录认证体系 | v1.2 | 0/3 | Not started | - |
|
||||||
|
| 8. 内容爬取实现 | v1.2 | 0/3 | Not started | - |
|
||||||
|
| 9. Pipeline 整合 | v1.2 | 0/1 | Not started | - |
|
||||||
|
|||||||
+12
-21
@@ -3,13 +3,13 @@ gsd_state_version: 1.0
|
|||||||
milestone: v1.2
|
milestone: v1.2
|
||||||
milestone_name: Zhihu Crawler
|
milestone_name: Zhihu Crawler
|
||||||
status: planning
|
status: planning
|
||||||
stopped_at: Defining requirements
|
stopped_at: Roadmap created
|
||||||
last_updated: "2026-03-17T10:00:00.000Z"
|
last_updated: "2026-03-17T12:00:00.000Z"
|
||||||
last_activity: 2026-03-17 — Milestone v1.2 started
|
last_activity: 2026-03-17 — v1.2 roadmap created
|
||||||
progress:
|
progress:
|
||||||
total_phases: 0
|
total_phases: 3
|
||||||
completed_phases: 0
|
completed_phases: 0
|
||||||
total_plans: 0
|
total_plans: 7
|
||||||
completed_plans: 0
|
completed_plans: 0
|
||||||
percent: 0
|
percent: 0
|
||||||
---
|
---
|
||||||
@@ -21,14 +21,14 @@ progress:
|
|||||||
See: .planning/PROJECT.md (updated 2026-03-17)
|
See: .planning/PROJECT.md (updated 2026-03-17)
|
||||||
|
|
||||||
**Core value:** 可靠、可维护的处理管道 - 每个步骤独立、可验证、可恢复
|
**Core value:** 可靠、可维护的处理管道 - 每个步骤独立、可验证、可恢复
|
||||||
**Current focus:** v1.2 Zhihu Crawler — 需求定义中
|
**Current focus:** v1.2 Zhihu Crawler — Phase 7 准备中
|
||||||
|
|
||||||
## Current Position
|
## Current Position
|
||||||
|
|
||||||
Phase: Not started (defining requirements)
|
Phase: 7 of 9 (登录认证体系)
|
||||||
Plan: —
|
Plan: —
|
||||||
Status: Defining requirements
|
Status: Ready to plan
|
||||||
Last activity: 2026-03-17 — Milestone v1.2 started
|
Last activity: 2026-03-17 — v1.2 roadmap created
|
||||||
|
|
||||||
Progress: [░░░░░░░░░░] 0% (v1.2 milestone)
|
Progress: [░░░░░░░░░░] 0% (v1.2 milestone)
|
||||||
|
|
||||||
@@ -54,15 +54,6 @@ v1.1 decisions:
|
|||||||
- 中间目录统一到 `~/work/intermediate/analysis/`
|
- 中间目录统一到 `~/work/intermediate/analysis/`
|
||||||
- 归档只保留 md 文件,不归档 json 文件
|
- 归档只保留 md 文件,不归档 json 文件
|
||||||
- 最终目录按平台分组,不再按时间创建子目录
|
- 最终目录按平台分组,不再按时间创建子目录
|
||||||
- [Phase 05]: Archive step is always last in pipeline, only md files archived
|
|
||||||
- [Phase 05]: Duplicate detection uses glob pattern *_{content_id}_analysis.md with user overwrite confirmation
|
|
||||||
- [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking
|
|
||||||
- [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level for tests
|
|
||||||
- [Phase 06]: Rename kb-workflow to kb_workflow for Python package compatibility
|
|
||||||
- [Phase 06]: Standard imports from kb_workflow.metadata instead of sys.path manipulation
|
|
||||||
- [Phase 06-code-consolidation]: Use absolute imports (kb_workflow.pipeline.xxx) for all pipeline modules
|
|
||||||
- [Phase 06-03]: External dependencies (kb-deep-analyzer) use lazy sys.path manipulation inside methods, not at module top-level
|
|
||||||
- [Phase 06-03]: Pipeline class exported at package level for cleaner imports
|
|
||||||
|
|
||||||
v1.2 decisions:
|
v1.2 decisions:
|
||||||
- 知乎登录策略:二维码扫码 + 手动 Cookie 降级
|
- 知乎登录策略:二维码扫码 + 手动 Cookie 降级
|
||||||
@@ -76,8 +67,8 @@ None.
|
|||||||
|
|
||||||
## Session Continuity
|
## Session Continuity
|
||||||
|
|
||||||
Last session: 2026-03-17T10:00:00.000Z
|
Last session: 2026-03-17T12:00:00.000Z
|
||||||
Stopped at: Defining requirements
|
Stopped at: Roadmap created
|
||||||
Resume file: None
|
Resume file: None
|
||||||
|
|
||||||
**Next step:** Define requirements → Create roadmap
|
**Next step:** Run `/gsd:plan-phase 7` to create first plan
|
||||||
|
|||||||
Reference in New Issue
Block a user