6.2 KiB
Roadmap: kb-skills
Milestones
- ✅ v1.0 Pipeline Architecture - Phases 1-4 (shipped 2026-03-15)
- ✅ v1.1 Architecture Refactor - Phases 5-6 (shipped 2026-03-16)
- ✅ v1.2 Zhihu Crawler - Phases 7-9 (shipped 2026-03-17)
- ✅ v1.3 Proxy Pool - Phase 10 (shipped 2026-03-18)
- ✅ v1.4 Zhihu Anti-Crawler - Phase 11 (shipped 2026-03-18)
- ✅ v1.5 B站字幕简化处理 - Phase 12 (shipped 2026-03-21)
- ✅ v1.6 语义分段改进 - Phases 13-15 (shipped 2026-03-23)
Phases
✅ v1.0 Pipeline Architecture (Phases 1-4) - SHIPPED 2026-03-15
Phase 1: Foundation
Goal: 项目基础设施和状态管理 Plans: 3 plans
Plans:
- 01-01: WorkflowStateManager 状态管理
- 01-02: LinkParser URL 解析
- 01-03: IdempotencyChecker 幂等检查
Phase 2: Adapters
Goal: 平台元数据适配器 Plans: 2 plans
Plans:
- 02-01: BilibiliAdapter B站适配器
- 02-02: ZhihuAdapter 知乎适配器
Phase 3: Processors
Goal: 管道处理器架构 Plans: 3 plans
Plans:
- 03-01: BaseProcessor 基类
- 03-02: 具体处理器实现
- 03-03: Schema 验证
Phase 4: Pipeline
Goal: 处理器编排和 CLI Plans: 2 plans
Plans:
- 04-01: Pipeline 编排引擎
- 04-02: CLI process 命令
✅ v1.1 Architecture Refactor (Phases 5-6) - SHIPPED 2026-03-16
Phase 5: 归档与目录优化
Goal: 统一中间目录结构，添加自动归档步骤，优化最终目录布局 Plans: 3 plans (all complete)
Phase 6: 代码整合
Goal: 将独立的元数据技能和 Pipeline 模块整合进 kb-workflow Plans: 3 plans (all complete)
✅ v1.2 Zhihu Crawler (Phases 7-9) - SHIPPED 2026-03-17
Phase 7: 登录认证体系
Goal: 建立知乎登录认证机制，支持二维码扫码和手动 Cookie 两种方式 Plans: 3 plans (all complete)
Phase 8: 内容爬取实现
Goal: 实现知乎问答、专栏/文章、评论的完整爬取功能 Plans: 3 plans (all complete)
Phase 9: Pipeline 整合
Goal: 将新爬虫实现整合进现有 Pipeline 架构 Plans: 1 plan (complete)
✅ v1.3 Proxy Pool (Phase 10) - SHIPPED 2026-03-18
Phase 10: Proxy Pool (代理池管理)
Goal: 建立代理池基础设施，支持多代理配置、自动轮换和健康检查 Requirements: PROXY-01, PROXY-02, PROXY-03, PROXY-04 Plans: 2 plans (all complete)
Plans:
- 10-01: ProxyPool 核心类实现 — 从 YAML/环境变量加载代理，Round-Robin 轮换
- 10-02: 健康检查与 ZhihuCrawler 集成 — 被动检测、冷却恢复、代理切换
Key Decisions:
- 使用索引指针而非 itertools.cycle 实现轮换
- 惰性冷却检查而非后台定时器
- ProxyPool 参数优先于 config.proxy 配置
✅ v1.4 Zhihu Anti-Crawler (Phase 11) - SHIPPED 2026-03-18
Phase 11: Stealth Fingerprinting (指纹伪装)
Goal: 隐藏 Playwright 自动化特征，让爬虫通过知乎的浏览器指纹检测 Depends on: Phase 10 Requirements: STEALTH-01, STEALTH-02, STEALTH-03, STEALTH-04, VERIFY-01, VERIFY-02 Plans: 2 plans (all complete)
Plans:
- 11-01: 创建 StealthConfig 模块和单元测试
- 11-02: 集成 stealth 到 ZhihuCrawler 并验证
Key Decisions:
- 使用 playwright-stealth 库进行指纹伪装
- stealth_config=None 表示默认启用 stealth (向后兼容)
- launch 时添加 --disable-blink-features=AutomationControlled 参数
✅ v1.5 B站字幕简化处理 (Phase 12) - SHIPPED 2026-03-21
Phase 12: SRT 简化处理器
Goal: 实现 SRT 字幕处理，生成简化的 Markdown 文档，保留元数据和标签，移除深度分析 Plans: 2 plans (all complete)
Plans:
- 12-01: SRT 处理模块（parser、segmenter、corrector）和 SRTProcessor 实现
- 12-02: Pipeline 集成、平台路由、测试验证
Key Decisions:
- SRT 处理模块独立于 DeepAnalyzer，位于 kb_workflow/srt/
- 语义分段基于简单规则（时间间隔 + 长度限制），不依赖 LLM
- ASR 纠错可选，AI 不可用时返回原文
- B 站视频使用 SRTProcessor，知乎/文章继续使用 AnalyzeProcessor
✅ v1.6 语义分段改进 (Phases 13-15) - SHIPPED 2026-03-23
完整归档信息: .planning/milestones/v1.6-ROADMAP.md
Phase 13: 核心 LLM 分段功能
Goal: 实现基于 LLM 的智能语义分段，识别视频类型并生成主题标题 Plans: 7 plans (all complete)
Phase 14: 鲁棒性与成本控制
Goal: 提高系统稳定性，实现优雅降级和成本优化 Plans: 6 plans (all complete)
Phase 15: 输出优化与兼容性
Goal: 优化输出格式，确保向后兼容和可配置性 Plans: 4 plans (all complete)
Delivered: 基于 LLM 的智能语义分段 — 支持视频类型识别、主题边界检测、标题生成，完全向后兼容
Progress
Execution Order: Phases execute in numeric order: 1 → 2 → ... → 15
| Phase | Milestone | Plans Complete | Status | Completed |
|---|---|---|---|---|
| 1. Foundation | v1.0 | 3/3 | Complete | 2026-03-15 |
| 2. Adapters | v1.0 | 2/2 | Complete | 2026-03-15 |
| 3. Processors | v1.0 | 3/3 | Complete | 2026-03-15 |
| 4. Pipeline | v1.0 | 2/2 | Complete | 2026-03-15 |
| 5. 归档与目录优化 | v1.1 | 3/3 | Complete | 2026-03-16 |
| 6. 代码整合 | v1.1 | 3/3 | Complete | 2026-03-16 |
| 7. 登录认证体系 | v1.2 | 3/3 | Complete | 2026-03-17 |
| 8. 内容爬取实现 | v1.2 | 3/3 | Complete | 2026-03-17 |
| 9. Pipeline 整合 | v1.2 | 1/1 | Complete | 2026-03-17 |
| 10. Proxy Pool | v1.3 | 2/2 | Complete | 2026-03-18 |
| 11. Stealth Fingerprinting | v1.4 | 2/2 | Complete | 2026-03-18 |
| 12. SRT 简化处理器 | v1.5 | 2/2 | Complete | 2026-03-21 |
| 13. 核心 LLM 分段功能 | v1.6 | 7/7 | Complete | 2026-03-22 |
| 14. 鲁棒性与成本控制 | v1.6 | 6/6 | Complete | 2026-03-23 |
| 15. 输出优化与兼容性 | v1.6 | 4/4 | Complete | 2026-03-23 |
Last updated: 2026-03-23 after v1.6 milestone