starmen1265/kb-skills
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
This repository has been archived on 2026-08-07. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
2cfdbd5aef4bad10593147b4501c5ae40ef3c2c2
kb-skills/.planning/milestones/v1.5-ROADMAP.md
T
starmen1265 c8dcc3e09b
PR Test / lint (pull_request) Successful in 2s
Details
chore: complete v1.5 milestone - B站字幕简化处理 
- Archive roadmap to milestones/v1.5-ROADMAP.md
- Archive requirements to milestones/v1.5-REQUIREMENTS.md
- Update ROADMAP.md with collapsed v1.5 entry
- Update PROJECT.md with validated requirements
- Add semantic segmentation to backlog

Delivered: B站视频简化处理流程

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 23:24:59 +08:00

6.4 KiB
Raw Blame History

Roadmap: kb-skills

Milestones

  • ✅ v1.0 Pipeline Architecture - Phases 1-4 (shipped 2026-03-15)
  • ✅ v1.1 Architecture Refactor - Phases 5-6 (shipped 2026-03-16)
  • ✅ v1.2 Zhihu Crawler - Phases 7-9 (shipped 2026-03-17)
  • ✅ v1.3 Proxy Pool - Phase 10 (shipped 2026-03-18)
  • ✅ v1.4 Zhihu Anti-Crawler - Phase 11 (shipped 2026-03-18)
  • ◆ v1.5 B站字幕简化处理 - Phase 12 (shipped 2026-03-19)

Phases

✅ v1.0 Pipeline Architecture (Phases 1-4) - SHIPPED 2026-03-15

Phase 1: Foundation

Goal: 项目基础设施和状态管理 Plans: 3 plans

Plans:

  • 01-01: WorkflowStateManager 状态管理
  • 01-02: LinkParser URL 解析
  • 01-03: IdempotencyChecker 幂等检查

Phase 2: Adapters

Goal: 平台元数据适配器 Plans: 2 plans

Plans:

  • 02-01: BilibiliAdapter B站适配器
  • 02-02: ZhihuAdapter 知乎适配器

Phase 3: Processors

Goal: 管道处理器架构 Plans: 3 plans

Plans:

  • 03-01: BaseProcessor 基类
  • 03-02: 具体处理器实现
  • 03-03: Schema 验证

Phase 4: Pipeline

Goal: 处理器编排和 CLI Plans: 2 plans

Plans:

  • 04-01: Pipeline 编排引擎
  • 04-02: CLI process 命令
✅ v1.1 Architecture Refactor (Phases 5-6) - SHIPPED 2026-03-16

Phase 5: 归档与目录优化

Goal: 统一中间目录结构，添加自动归档步骤，优化最终目录布局 Plans: 3 plans (all complete)

Phase 6: 代码整合

Goal: 将独立的元数据技能和 Pipeline 模块整合进 kb-workflow Plans: 3 plans (all complete)

✅ v1.2 Zhihu Crawler (Phases 7-9) - SHIPPED 2026-03-17

Phase 7: 登录认证体系

Goal: 建立知乎登录认证机制，支持二维码扫码和手动 Cookie 两种方式 Plans: 3 plans (all complete)

Phase 8: 内容爬取实现

Goal: 实现知乎问答、专栏/文章、评论的完整爬取功能 Plans: 3 plans (all complete)

Phase 9: Pipeline 整合

Goal: 将新爬虫实现整合进现有 Pipeline 架构 Plans: 1 plan (complete)

✅ v1.3 Proxy Pool (Phase 10) - SHIPPED 2026-03-18

Phase 10: Proxy Pool (代理池管理)

Goal: 建立代理池基础设施，支持多代理配置、自动轮换和健康检查 Requirements: PROXY-01, PROXY-02, PROXY-03, PROXY-04 Plans: 2 plans (all complete)

Plans:

  • 10-01: ProxyPool 核心类实现 — 从 YAML/环境变量加载代理，Round-Robin 轮换
  • 10-02: 健康检查与 ZhihuCrawler 集成 — 被动检测、冷却恢复、代理切换

Key Decisions:

  • 使用索引指针而非 itertools.cycle 实现轮换
  • 惰性冷却检查而非后台定时器
  • ProxyPool 参数优先于 config.proxy 配置

✅ v1.4 Zhihu Anti-Crawler (SHIPPED 2026-03-18)

Milestone Goal: 绕过知乎反爬检测，让 Playwright 爬虫能正常工作

Phase 11: Stealth Fingerprinting (指纹伪装)

Goal: 隐藏 Playwright 自动化特征，让爬虫通过知乎的浏览器指纹检测 Depends on: Phase 10 Requirements: STEALTH-01, STEALTH-02, STEALTH-03, STEALTH-04, VERIFY-01, VERIFY-02 Success Criteria (what must be TRUE):

  1. ZhihuCrawler 访问问答页面时不再被 40362 错误拦截
  2. 爬虫的 navigator.webdriver 属性对知乎不可见
  3. Playwright 特有的 JS 属性（__playwright, pwInspect 等）被移除
  4. 现有功能（登录、Cookie 持久化、内容爬取）保持正常工作 Plans: 2 plans (all complete)

Plans:

  • 11-01-PLAN.md — 创建 StealthConfig 模块和单元测试
  • 11-02-PLAN.md — 集成 stealth 到 ZhihuCrawler 并验证

Key Decisions:

  • 使用 playwright-stealth 库进行指纹伪装
  • stealth_config=None 表示默认启用 stealth (向后兼容)
  • launch 时添加 --disable-blink-features=AutomationControlled 参数

✅ v1.5 B站字幕简化处理 (SHIPPED 2026-03-19)

Milestone Goal: 简化 B 站视频处理流程，基于 SRT 字幕生成易读的 Markdown 文档

Phase 12: SRT 简化处理器

Goal: 实现 SRT 字幕处理，生成简化的 Markdown 文档，保留元数据和标签，移除深度分析 Depends on: Phase 4 (Pipeline) Requirements: SRT-01, SRT-02, SRT-03, SRT-04, OUT-01, OUT-02, OUT-03, OUT-04, SCOPE-01, SCOPE-02 Success Criteria (what must be TRUE):

  1. 用户运行 kb-workflow process <bilibili-url> 后生成 Markdown 文件而非深度分析报告
  2. Markdown 文件包含视频元数据（标题、作者、时长）
  3. 元数据处理功能保持不变（BilibiliAdapter 正常工作）
  4. 视频标签继续正常生成
  5. 字幕文本已去除时间戳，按语义分段
  6. 明显的语音识别错误已被 LLM 纠正
  7. DeepAnalyzer 相关代码已移除或禁用
  8. 知乎等其他平台处理器不受影响 Plans: 2 plans (all complete)

Plans:

  • 12-01-PLAN.md — SRT 处理模块（parser、segmenter、corrector）和 SRTProcessor 实现
  • 12-02-PLAN.md — Pipeline 集成、平台路由、测试验证

Key Decisions:

  • SRT 处理模块独立于 DeepAnalyzer，位于 kb_workflow/srt/
  • 语义分段基于简单规则（时间间隔 + 长度限制），不依赖 LLM
  • ASR 纠错可选，AI 不可用时返回原文
  • B 站视频使用 SRTProcessor，知乎/文章继续使用 AnalyzeProcessor
  • DeepAnalyzer 保留但添加弃用警告（仅针对 B 站视频）

Progress

Execution Order: Phases execute in numeric order: 1 → 2 → ... → 10 → 11 → 12

Phase Milestone Plans Complete Status Completed
1. Foundation v1.0 3/3 Complete 2026-03-15
2. Adapters v1.0 2/2 Complete 2026-03-15
3. Processors v1.0 3/3 Complete 2026-03-15
4. Pipeline v1.0 2/2 Complete 2026-03-15
5. 归档与目录优化 v1.1 3/3 Complete 2026-03-16
6. 代码整合 v1.1 3/3 Complete 2026-03-16
7. 登录认证体系 v1.2 3/3 Complete 2026-03-17
8. 内容爬取实现 v1.2 3/3 Complete 2026-03-17
9. Pipeline 整合 v1.2 1/1 Complete 2026-03-17
10. Proxy Pool v1.3 2/2 Complete 2026-03-18
11. Stealth Fingerprinting v1.4 2/2 Complete 2026-03-18
12. SRT 简化处理器 v1.5 Complete 2026-03-19 2026-03-19

Last updated: 2026-03-19

Reference in New Issue View Git Blame Copy Permalink