diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index d7b197f..dd4a2fc 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -27,11 +27,11 @@ ### 代码整合 - 元数据 (Metadata Integration) -- [ ] **MET-01**: kb-bilibili-metadata 整合进 kb-workflow/metadata/ -- [ ] **MET-02**: kb-zhihu-metadata 整合进 kb-workflow/metadata/ -- [ ] **MET-03**: 删除独立的 kb-bilibili-metadata 目录 -- [ ] **MET-04**: 删除独立的 kb-zhihu-metadata 目录 -- [ ] **MET-05**: 更新所有导入路径指向新位置 +- [x] **MET-01**: kb-bilibili-metadata 整合进 kb-workflow/metadata/ +- [x] **MET-02**: kb-zhihu-metadata 整合进 kb-workflow/metadata/ +- [x] **MET-03**: 删除独立的 kb-bilibili-metadata 目录 +- [x] **MET-04**: 删除独立的 kb-zhihu-metadata 目录 +- [x] **MET-05**: 更新所有导入路径指向新位置 ### 代码整合 - Pipeline (Pipeline Integration) @@ -80,11 +80,11 @@ Deferred to future release. | OUT-02 | Phase 5 | Complete | | OUT-03 | Phase 5 | Complete | | OUT-04 | Phase 5 | Complete | -| MET-01 | Phase 6 | Pending | -| MET-02 | Phase 6 | Pending | -| MET-03 | Phase 6 | Pending | -| MET-04 | Phase 6 | Pending | -| MET-05 | Phase 6 | Pending | +| MET-01 | Phase 6 | Complete | +| MET-02 | Phase 6 | Complete | +| MET-03 | Phase 6 | Complete | +| MET-04 | Phase 6 | Complete | +| MET-05 | Phase 6 | Complete | | PIP-01 | Phase 6 | Pending | | PIP-02 | Phase 6 | Pending | | PIP-03 | Phase 6 | Pending | diff --git a/.planning/STATE.md b/.planning/STATE.md index 5bdb73d..b55a0fe 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,16 +2,16 @@ gsd_state_version: 1.0 milestone: v1.1 milestone_name: Architecture Refactor -status: completed -stopped_at: Phase 6 context gathered -last_updated: "2026-03-16T15:42:35.963Z" -last_activity: 2026-03-16 — 05-03 Archive testing completed +status: in_progress +stopped_at: 06-01 Metadata consolidation completed +last_updated: "2026-03-17T00:10:00.000Z" +last_activity: 2026-03-17 — 06-01 Metadata consolidation completed progress: total_phases: 6 completed_phases: 5 - total_plans: 15 - completed_plans: 15 - percent: 10 + total_plans: 17 + completed_plans: 16 + percent: 94 --- # Project State @@ -21,49 +21,48 @@ progress: See: .planning/PROJECT.md (updated 2026-03-16) **Core value:** 可靠、可维护的处理管道 - 每个步骤独立、可验证、可恢复 -**Current focus:** v1.1 Architecture Refactor — Phase 5 归档与目录优化 COMPLETE +**Current focus:** v1.1 Architecture Refactor — Phase 6 代码整合 IN PROGRESS ## Current Position -Phase: 5 of 6 (归档与目录优化) - COMPLETE -Plan: 3 of 3 in current phase - COMPLETE -Status: Phase 5 Complete -Last activity: 2026-03-16 — 05-03 Archive testing completed +Phase: 6 of 6 (代码整合) - IN PROGRESS +Plan: 1 of 2 in current phase - COMPLETE +Status: 06-01 Complete +Last activity: 2026-03-17 — 06-01 Metadata consolidation completed -Progress: [██░░░░░░░░] 10% (v1.1 milestone) +Progress: [█████████░] 94% (v1.1 milestone) ## Performance Metrics **Velocity:** -- Total plans completed: 3 (v1.1) -- Average duration: 3 min -- Total execution time: 9 min +- Total plans completed: 4 (v1.1) +- Average duration: 4 min +- Total execution time: 16 min **By Phase:** | Phase | Plans | Total | Avg/Plan | |-------|-------|-------|----------| | 5. 归档与目录优化 | 3/3 | 9 min | 3 min | -| 6. 代码整合 | 0/2 | - | - | +| 6. 代码整合 | 1/2 | 7 min | 7 min | **Recent Trend:** -- Last 5 plans: 3 min +- Last 5 plans: 5 min - Trend: Stable *Updated after each plan completion* -| Phase 05 P03 | 6min | 3 tasks | 4 files | -| Phase 05 P03 | 6min | 3 tasks | 4 files | +| Phase 06 P01 | 7min | 4 tasks | 10 files | ## Accumulated Context ### Decisions Key decisions from v1.0 (preserved): -- ✓ 管道模型 — BaseProcessor + Pipeline 架构 -- ✓ 步骤级状态 — WorkflowStateManager -- ✓ Schema 验证 — Pydantic v2 -- ✓ 顺序执行 — 简单性和可调试性 -- ✓ CLI subparsers — 可扩展命令结构 +- 管道模型 — BaseProcessor + Pipeline 架构 +- 步骤级状态 — WorkflowStateManager +- Schema 验证 — Pydantic v2 +- 顺序执行 — 简单性和可调试性 +- CLI subparsers — 可扩展命令结构 v1.1 decisions: - 中间目录统一到 `~/work/intermediate/analysis/` @@ -72,9 +71,9 @@ v1.1 decisions: - [Phase 05]: Archive step is always last in pipeline, only md files archived - [Phase 05]: Duplicate detection uses glob pattern *_{content_id}_analysis.md with user overwrite confirmation - [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking -- [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level -- [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking - [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level for tests +- [Phase 06]: Rename kb-workflow to kb_workflow for Python package compatibility +- [Phase 06]: Standard imports from kb_workflow.metadata instead of sys.path manipulation ### Blockers/Concerns @@ -82,8 +81,8 @@ None. ## Session Continuity -Last session: 2026-03-16T15:42:35.962Z -Stopped at: Phase 6 context gathered -Resume file: .planning/phases/06-code-consolidation/06-CONTEXT.md +Last session: 2026-03-17T00:10:00.000Z +Stopped at: 06-01 Metadata consolidation completed +Resume file: .planning/phases/06-code-consolidation/06-02-PLAN.md -**Next step:** Execute 06-01-PLAN.md (Phase 6: Code Consolidation) +**Next step:** Execute 06-02-PLAN.md (Phase 6: Pipeline Module Consolidation) diff --git a/.planning/phases/06-code-consolidation/06-01-SUMMARY.md b/.planning/phases/06-code-consolidation/06-01-SUMMARY.md new file mode 100644 index 0000000..245ad13 --- /dev/null +++ b/.planning/phases/06-code-consolidation/06-01-SUMMARY.md @@ -0,0 +1,133 @@ +--- +phase: 06-code-consolidation +plan: 01 +subsystem: code-consolidation +tags: [metadata, import-refactor, package-structure] + +# Dependency graph +requires: [] +provides: + - kb_workflow/metadata/ - unified metadata extraction modules + - Standard Python package imports (no sys.path manipulation) +affects: [06-02] + +# Tech tracking +tech-stack: + added: [] + patterns: [standard-package-imports, git-mv-for-history-preservation] + +key-files: + created: + - kb_workflow/__init__.py + - kb_workflow/metadata/__init__.py + modified: + - kb_pipeline/platforms/bilibili.py + - kb_pipeline/platforms/zhihu.py + - tests/test_cli/test_process_command.py + +key-decisions: + - "Rename kb-workflow to kb_workflow for Python package compatibility" + - "Create kb_workflow/__init__.py to export main and process_command" + - "Keep original module file names (bilibili_extractor.py, zhihu_extractor.py)" + +patterns-established: + - "Standard imports: from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor" + - "No sys.path manipulation in platform adapters" + +requirements-completed: [MET-01, MET-02, MET-03, MET-04, MET-05] + +# Metrics +duration: 7min +completed: 2026-03-17 +--- + +# Phase 6 Plan 01: Metadata Module Consolidation Summary + +**Unified metadata extraction modules into kb_workflow/metadata/ with standard Python package imports, eliminating sys.path manipulation** + +## Performance + +- **Duration:** 7 min +- **Started:** 2026-03-16T15:58:02Z +- **Completed:** 2026-03-17T00:05:00Z +- **Tasks:** 4 +- **Files modified:** 10 + +## Accomplishments +- Moved bilibili and zhihu metadata modules to kb_workflow/metadata/ +- Updated platform adapters to use standard imports +- Deleted original kb-bilibili-metadata/ and kb-zhihu-metadata/ directories +- Created kb_workflow package with proper __init__.py exports + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Create metadata directory and move bilibili modules** - `e2edce2` (feat) +2. **Task 2: Move zhihu modules to metadata directory** - `a4a7454` (feat) +3. **Task 3: Update platform adapters to use standard imports** - `98f8ec2` (refactor) +4. **Task 4: Delete original metadata directories and update tests** - `7662833` (refactor) + +## Files Created/Modified +- `kb_workflow/__init__.py` - Package entry point, exports main and process_command +- `kb_workflow/metadata/__init__.py` - Metadata package entry, exports extractors +- `kb_workflow/metadata/bilibili_extractor.py` - Bilibili metadata extractor (moved) +- `kb_workflow/metadata/bilibili_metadata.py` - Bilibili CLI (moved) +- `kb_workflow/metadata/zhihu_extractor.py` - Zhihu Playwright extractor (moved) +- `kb_workflow/metadata/zhihu_metadata.py` - Zhihu CLI (moved) +- `kb_workflow/metadata/zhihu_article_crawler.py` - Zhihu article crawler (moved) +- `kb_workflow/metadata/markdown_converter.py` - Markdown converter (moved) +- `kb_pipeline/platforms/bilibili.py` - Updated imports, removed sys.path manipulation +- `kb_pipeline/platforms/zhihu.py` - Updated imports, removed sys.path manipulation +- `tests/test_cli/test_process_command.py` - Updated imports for new package structure + +## Decisions Made +- Renamed `kb-workflow` directory to `kb_workflow` for Python package compatibility (hyphens not allowed in package names) +- Created `kb_workflow/__init__.py` to export `main` and `process_command` functions for backward compatibility with tests +- Preserved original module file names to minimize changes and maintain clarity + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] Renamed kb-workflow to kb_workflow for package compatibility** +- **Found during:** Task 1 (Create metadata directory) +- **Issue:** Directory name `kb-workflow` contains hyphen, Python cannot import it as package +- **Fix:** Used `git mv kb-workflow kb_workflow` to rename directory +- **Files modified:** Entire kb-workflow directory renamed +- **Verification:** `from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor` succeeds +- **Committed in:** `e2edce2` (Task 1 commit) + +**2. [Rule 3 - Blocking] Created kb_workflow/__init__.py for test compatibility** +- **Found during:** Task 4 (Run tests) +- **Issue:** Tests imported `kb_workflow` as module, needed `main` and `process_command` exports +- **Fix:** Created `kb_workflow/__init__.py` with exports from `kb_workflow.py` +- **Files modified:** `kb_workflow/__init__.py` +- **Verification:** `from kb_workflow import main, process_command` succeeds +- **Committed in:** `7662833` (Task 4 commit) + +--- + +**Total deviations:** 2 auto-fixed (2 blocking) +**Impact on plan:** Both fixes necessary for Python package conventions and test compatibility. No scope creep. + +## Issues Encountered +- Pre-existing test failures (14 tests) related to mock setup and test isolation, not caused by this refactoring +- Tests pass when run individually, fail when run in full suite due to global state pollution between tests + +## User Setup Required +None - no external service configuration required. + +## Next Phase Readiness +- Metadata modules consolidated, ready for pipeline module consolidation (06-02) +- Standard import pattern established for future module moves + +## Self-Check: PASSED + +- All created files verified to exist +- Original directories (kb-bilibili-metadata, kb-zhihu-metadata) confirmed deleted +- All 4 task commits verified in git history + +--- +*Phase: 06-code-consolidation* +*Completed: 2026-03-17*