Commit Graph

121 Commits

Author SHA1 Message Date
starmen1265 248a1f2b8c docs(09-01): complete ZhihuAdapter integration plan
- Integrated ZhihuCrawler into ZhihuAdapter
- Added comment crawling and formatting
- Implemented cookie auto-login flow
- Added idempotent execution via IdempotencyChecker

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 23:17:39 +08:00
starmen1265 22ed60a5ab docs(09-00): complete Wave 0 test scaffolding plan
- Added 5 mock fixtures for ZhihuCrawler integration testing
- Added 6 test skeletons (marked skip) for Wave 1 implementation
- Updated STATE.md with Phase 9 position
- Updated ROADMAP.md with plan progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 23:02:40 +08:00
starmen1265 bee696b96d docs(09): complete phase 9 planning with 2 plans in 2 waves 2026-03-17 22:54:48 +08:00
starmen1265 1c42443dce fix(09): revise plans based on checker feedback 2026-03-17 22:53:00 +08:00
starmen1265 ec18ea9197 docs(09): create phase 9 plans 2026-03-17 22:46:21 +08:00
starmen1265 45ba9c973f docs(09): research phase 9 domain 2026-03-17 22:37:38 +08:00
starmen1265 b3c7386098 docs(state): record phase 9 context session 2026-03-17 22:32:56 +08:00
starmen1265 e6b62f42e7 docs(09): capture phase context 2026-03-17 22:32:44 +08:00
starmen1265 007b0d70ef docs(08): complete phase 08 - Zhihu Crawler
Phase 08: Crawler - All 3 plans complete

Verification: 7/7 must-haves passed
Tests: 44 passed

Requirements completed:
- CONT-01: Crawl Zhihu answers
- CONT-02: Crawl Zhihu articles
- CONT-03: Crawl comments (optional)
- CONT-04: Metadata extraction
- NET-01: Proxy configuration
- NET-02: Mandatory proxy
- NET-03: Rate limiting & retry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 22:23:56 +08:00
starmen1265 123e307522 docs(08-03): complete anti-crawl measures plan
- Proxy configuration from HTTP_PROXY/HTTPS_PROXY
- Random delay (1-3s) between requests
- Exponential backoff retry (3 attempts)
- x-zse-96 signature handled by browser

All 44 crawler tests passing

Requirements: NET-01, NET-02, NET-03 completed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 22:20:38 +08:00
starmen1265 6a4f13b2f6 docs(08-02): complete comment crawler plan
- Add 08-02-SUMMARY.md with execution details
- Update STATE.md with decisions and metrics
- Mark CONT-03 requirement complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 21:59:49 +08:00
starmen1265 f0ee4ca5e2 docs(08-01): complete zhihu content crawler plan
- ZhihuCrawler class with API interception
- AnswerData, ArticleData models
- 15 tests passing

Requirements: CONT-01, CONT-02, CONT-04 completed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 21:51:56 +08:00
starmen1265 334aac87c7 fix(08): resolve Wave 1 file conflict - move 08-02 to Wave 2 2026-03-17 21:02:17 +08:00
starmen1265 6d2d6b26e9 docs(08): create phase 8 plans for crawler implementation 2026-03-17 20:56:51 +08:00
starmen1265 c93237abde docs(08): add validation strategy 2026-03-17 20:49:01 +08:00
starmen1265 1464c9b707 docs(08): research crawler implementation domain 2026-03-17 20:47:12 +08:00
starmen1265 ba33e0e518 docs(state): record phase 08 context session 2026-03-17 18:57:06 +08:00
starmen1265 87adf8aca6 docs(08): capture phase context for crawler implementation 2026-03-17 18:56:56 +08:00
starmen1265 e8eaca519f docs(07-03): complete Login CLI Subcommand plan
- Cookie validation via /api/v4/me endpoint
- login CLI subcommand with 3 modes: scan, import, status
- AUTH-04 requirement complete: Cookie expiry detection
- Phase 7 Login Auth complete, ready for Phase 8 Crawler

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 18:24:32 +08:00
starmen1265 4228b1e3a1 docs(07-01): complete QR Code Login Core plan 2026-03-17 17:57:28 +08:00
starmen1265 799bf09acc docs(07-02): add self-check verification to summary
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:54:12 +08:00
starmen1265 771ca554b3 docs(07-02): complete cookie import & persistence plan 2026-03-17 17:53:38 +08:00
starmen1265 d6a8690bc7 docs(07): add phase plans (3 plans, 2 waves) 2026-03-17 17:18:12 +08:00
starmen1265 14434df4eb docs(07): create phase 7 login auth plans 2026-03-17 17:15:30 +08:00
starmen1265 1d1f3309b2 docs(07): research phase domain - login auth system 2026-03-17 17:08:30 +08:00
starmen1265 58487b0fa3 docs(state): record phase 7 context session 2026-03-17 16:57:14 +08:00
starmen1265 87c7e50c4f docs(07): capture phase context 2026-03-17 16:57:02 +08:00
starmen1265 367b2816c4 docs: create milestone v1.2 roadmap (3 phases) 2026-03-17 16:31:17 +08:00
starmen1265 dcba34b643 docs: define milestone v1.2 requirements 2026-03-17 16:27:05 +08:00
starmen1265 9077a783e8 docs: start milestone v1.2 Zhihu Crawler 2026-03-17 16:16:38 +08:00
starmen1265 2dce679cca fix(debug): resolve kb-pipeline import issue in skill directory
PR Test / lint (pull_request) Successful in 2s
Root cause: ~/.openclaw/skills/kb-workflow/kb_workflow.py used old
import paths (kb_pipeline.*) after v1.1 refactor renamed package to
kb_workflow.pipeline

Fix applied to external skill directory:
- Updated imports from kb_pipeline.* to kb_workflow.pipeline.*
- Fixed project_root path calculation
- Updated SKILL.md documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 11:01:33 +08:00
starmen1265 8d67f3c953 docs(06): complete phase 6 with verification passed
PR Test / lint (pull_request) Successful in 1s
- All 3 plans executed (06-01, 06-02, 06-03)
- sys.path operations eliminated
- Pipeline exported at package level
- Phase goal achieved: clean Python package structure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:40:01 +08:00
starmen1265 2bec25e15c docs(06-03): complete gap closure plan 2026-03-17 00:37:30 +08:00
starmen1265 fdd6c08211 docs(06): create gap closure plan 06-03 2026-03-17 00:28:35 +08:00
starmen1265 3edbdcc183 docs(06): add phase verification report
PR Test / lint (pull_request) Successful in 2s
9/9 core functions verified, but gaps found:
- 4 remaining sys.path manipulations
- Pipeline not exported at package level

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:24:34 +08:00
starmen1265 190a04068b docs(06-02): complete pipeline module consolidation plan
- Move kb_pipeline/ to kb_workflow.pipeline/
- Update all imports to use absolute paths
- 278 tests pass (14 pre-existing failures unrelated to refactoring)
- CLI works with new module structure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:18:06 +08:00
starmen1265 65fdf4646a docs(06-01): complete metadata consolidation plan 2026-03-17 00:08:09 +08:00
starmen1265 444ac22e44 docs(06): create phase 6 consolidation plans 2026-03-16 23:52:44 +08:00
starmen1265 5fc7000ef7 docs(06): add research and validation strategy 2026-03-16 23:47:46 +08:00
starmen1265 33acdf0c2e docs(phase-06): research code consolidation domain 2026-03-16 23:46:37 +08:00
starmen1265 24806adb90 docs(state): record phase 6 context session 2026-03-16 23:42:40 +08:00
starmen1265 9f4aee0ac7 docs(06): capture phase context 2026-03-16 23:42:31 +08:00
starmen1265 bee61f6e08 docs(phase-05): complete phase execution - archive-directory
- Phase 5 归档与目录优化完成
- 所有 11 个需求 (INT/ARC/OUT) 已验证通过
- 测试: 8 个归档相关测试通过

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:36:14 +08:00
starmen1265 ea672f4526 docs(05-03): complete archive testing plan
- Add SUMMARY.md with 8 tests (5 unit + 3 integration)
- Update STATE.md: Phase 5 complete, advance to Phase 6
- Mark requirements OUT-01, OUT-02, OUT-03, OUT-04 complete
- Document test patterns and decisions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:31:47 +08:00
starmen1265 ccfa201250 fix(05-03): fix pre-existing test issues blocking verification
- Fix test_workflow_skips_completed_steps to mock _ask_rerun_step
- Fix test_fetch_video_metadata_success to use AsyncMock
- Document other pre-existing test issues in deferred-items.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:28:27 +08:00
starmen1265 bd5ffab6e7 docs(05-02): complete ArchiveProcessor implementation plan 2026-03-16 23:21:06 +08:00
starmen1265 129a6311f4 docs(05-01): complete unified intermediate directory plan
- Added intermediate_path config field with KB_INTERMEDIATE_PATH env var support
- Updated Pipeline to use configurable intermediate directory
- Updated CLI to default to intermediate_path instead of cwd
- Completed requirements: INT-01, INT-02, INT-03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:16:09 +08:00
starmen1265 18bfb8886d docs(05): add INT-03 to Phase 5 requirements
- Update ROADMAP.md to include INT-03 (configurable intermediate path)
- Update 05-01 plan description to reflect INT-03 coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:10:50 +08:00
starmen1265 ae69878b59 fix(05): revise plan to include KB_INTERMEDIATE_PATH config support
- Add INT-03 requirement to 05-01-PLAN.md requirements field
- Add Task 1 for KB_INTERMEDIATE_PATH config in KBSettings
- Update Task 2 and Task 3 to use configurable path instead of hardcoded
- Update 05-CONTEXT.md to reflect new decision (config support vs hardcoded)

Addresses checker issue: requirement_coverage for INT-03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:08:27 +08:00
starmen1265 a11a2dde99 docs(05): create phase 5 plans for archive and directory optimization 2026-03-16 23:03:14 +08:00