docs(06-01): complete metadata consolidation plan
This commit is contained in:
+10
-10
@@ -27,11 +27,11 @@
|
||||
|
||||
### 代码整合 - 元数据 (Metadata Integration)
|
||||
|
||||
- [ ] **MET-01**: kb-bilibili-metadata 整合进 kb-workflow/metadata/
|
||||
- [ ] **MET-02**: kb-zhihu-metadata 整合进 kb-workflow/metadata/
|
||||
- [ ] **MET-03**: 删除独立的 kb-bilibili-metadata 目录
|
||||
- [ ] **MET-04**: 删除独立的 kb-zhihu-metadata 目录
|
||||
- [ ] **MET-05**: 更新所有导入路径指向新位置
|
||||
- [x] **MET-01**: kb-bilibili-metadata 整合进 kb-workflow/metadata/
|
||||
- [x] **MET-02**: kb-zhihu-metadata 整合进 kb-workflow/metadata/
|
||||
- [x] **MET-03**: 删除独立的 kb-bilibili-metadata 目录
|
||||
- [x] **MET-04**: 删除独立的 kb-zhihu-metadata 目录
|
||||
- [x] **MET-05**: 更新所有导入路径指向新位置
|
||||
|
||||
### 代码整合 - Pipeline (Pipeline Integration)
|
||||
|
||||
@@ -80,11 +80,11 @@ Deferred to future release.
|
||||
| OUT-02 | Phase 5 | Complete |
|
||||
| OUT-03 | Phase 5 | Complete |
|
||||
| OUT-04 | Phase 5 | Complete |
|
||||
| MET-01 | Phase 6 | Pending |
|
||||
| MET-02 | Phase 6 | Pending |
|
||||
| MET-03 | Phase 6 | Pending |
|
||||
| MET-04 | Phase 6 | Pending |
|
||||
| MET-05 | Phase 6 | Pending |
|
||||
| MET-01 | Phase 6 | Complete |
|
||||
| MET-02 | Phase 6 | Complete |
|
||||
| MET-03 | Phase 6 | Complete |
|
||||
| MET-04 | Phase 6 | Complete |
|
||||
| MET-05 | Phase 6 | Complete |
|
||||
| PIP-01 | Phase 6 | Pending |
|
||||
| PIP-02 | Phase 6 | Pending |
|
||||
| PIP-03 | Phase 6 | Pending |
|
||||
|
||||
+30
-31
@@ -2,16 +2,16 @@
|
||||
gsd_state_version: 1.0
|
||||
milestone: v1.1
|
||||
milestone_name: Architecture Refactor
|
||||
status: completed
|
||||
stopped_at: Phase 6 context gathered
|
||||
last_updated: "2026-03-16T15:42:35.963Z"
|
||||
last_activity: 2026-03-16 — 05-03 Archive testing completed
|
||||
status: in_progress
|
||||
stopped_at: 06-01 Metadata consolidation completed
|
||||
last_updated: "2026-03-17T00:10:00.000Z"
|
||||
last_activity: 2026-03-17 — 06-01 Metadata consolidation completed
|
||||
progress:
|
||||
total_phases: 6
|
||||
completed_phases: 5
|
||||
total_plans: 15
|
||||
completed_plans: 15
|
||||
percent: 10
|
||||
total_plans: 17
|
||||
completed_plans: 16
|
||||
percent: 94
|
||||
---
|
||||
|
||||
# Project State
|
||||
@@ -21,49 +21,48 @@ progress:
|
||||
See: .planning/PROJECT.md (updated 2026-03-16)
|
||||
|
||||
**Core value:** 可靠、可维护的处理管道 - 每个步骤独立、可验证、可恢复
|
||||
**Current focus:** v1.1 Architecture Refactor — Phase 5 归档与目录优化 COMPLETE
|
||||
**Current focus:** v1.1 Architecture Refactor — Phase 6 代码整合 IN PROGRESS
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: 5 of 6 (归档与目录优化) - COMPLETE
|
||||
Plan: 3 of 3 in current phase - COMPLETE
|
||||
Status: Phase 5 Complete
|
||||
Last activity: 2026-03-16 — 05-03 Archive testing completed
|
||||
Phase: 6 of 6 (代码整合) - IN PROGRESS
|
||||
Plan: 1 of 2 in current phase - COMPLETE
|
||||
Status: 06-01 Complete
|
||||
Last activity: 2026-03-17 — 06-01 Metadata consolidation completed
|
||||
|
||||
Progress: [██░░░░░░░░] 10% (v1.1 milestone)
|
||||
Progress: [█████████░] 94% (v1.1 milestone)
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
**Velocity:**
|
||||
- Total plans completed: 3 (v1.1)
|
||||
- Average duration: 3 min
|
||||
- Total execution time: 9 min
|
||||
- Total plans completed: 4 (v1.1)
|
||||
- Average duration: 4 min
|
||||
- Total execution time: 16 min
|
||||
|
||||
**By Phase:**
|
||||
|
||||
| Phase | Plans | Total | Avg/Plan |
|
||||
|-------|-------|-------|----------|
|
||||
| 5. 归档与目录优化 | 3/3 | 9 min | 3 min |
|
||||
| 6. 代码整合 | 0/2 | - | - |
|
||||
| 6. 代码整合 | 1/2 | 7 min | 7 min |
|
||||
|
||||
**Recent Trend:**
|
||||
- Last 5 plans: 3 min
|
||||
- Last 5 plans: 5 min
|
||||
- Trend: Stable
|
||||
|
||||
*Updated after each plan completion*
|
||||
| Phase 05 P03 | 6min | 3 tasks | 4 files |
|
||||
| Phase 05 P03 | 6min | 3 tasks | 4 files |
|
||||
| Phase 06 P01 | 7min | 4 tasks | 10 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
### Decisions
|
||||
|
||||
Key decisions from v1.0 (preserved):
|
||||
- ✓ 管道模型 — BaseProcessor + Pipeline 架构
|
||||
- ✓ 步骤级状态 — WorkflowStateManager
|
||||
- ✓ Schema 验证 — Pydantic v2
|
||||
- ✓ 顺序执行 — 简单性和可调试性
|
||||
- ✓ CLI subparsers — 可扩展命令结构
|
||||
- 管道模型 — BaseProcessor + Pipeline 架构
|
||||
- 步骤级状态 — WorkflowStateManager
|
||||
- Schema 验证 — Pydantic v2
|
||||
- 顺序执行 — 简单性和可调试性
|
||||
- CLI subparsers — 可扩展命令结构
|
||||
|
||||
v1.1 decisions:
|
||||
- 中间目录统一到 `~/work/intermediate/analysis/`
|
||||
@@ -72,9 +71,9 @@ v1.1 decisions:
|
||||
- [Phase 05]: Archive step is always last in pipeline, only md files archived
|
||||
- [Phase 05]: Duplicate detection uses glob pattern *_{content_id}_analysis.md with user overwrite confirmation
|
||||
- [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking
|
||||
- [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level
|
||||
- [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking
|
||||
- [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level for tests
|
||||
- [Phase 06]: Rename kb-workflow to kb_workflow for Python package compatibility
|
||||
- [Phase 06]: Standard imports from kb_workflow.metadata instead of sys.path manipulation
|
||||
|
||||
### Blockers/Concerns
|
||||
|
||||
@@ -82,8 +81,8 @@ None.
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-03-16T15:42:35.962Z
|
||||
Stopped at: Phase 6 context gathered
|
||||
Resume file: .planning/phases/06-code-consolidation/06-CONTEXT.md
|
||||
Last session: 2026-03-17T00:10:00.000Z
|
||||
Stopped at: 06-01 Metadata consolidation completed
|
||||
Resume file: .planning/phases/06-code-consolidation/06-02-PLAN.md
|
||||
|
||||
**Next step:** Execute 06-01-PLAN.md (Phase 6: Code Consolidation)
|
||||
**Next step:** Execute 06-02-PLAN.md (Phase 6: Pipeline Module Consolidation)
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
---
|
||||
phase: 06-code-consolidation
|
||||
plan: 01
|
||||
subsystem: code-consolidation
|
||||
tags: [metadata, import-refactor, package-structure]
|
||||
|
||||
# Dependency graph
|
||||
requires: []
|
||||
provides:
|
||||
- kb_workflow/metadata/ - unified metadata extraction modules
|
||||
- Standard Python package imports (no sys.path manipulation)
|
||||
affects: [06-02]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: [standard-package-imports, git-mv-for-history-preservation]
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- kb_workflow/__init__.py
|
||||
- kb_workflow/metadata/__init__.py
|
||||
modified:
|
||||
- kb_pipeline/platforms/bilibili.py
|
||||
- kb_pipeline/platforms/zhihu.py
|
||||
- tests/test_cli/test_process_command.py
|
||||
|
||||
key-decisions:
|
||||
- "Rename kb-workflow to kb_workflow for Python package compatibility"
|
||||
- "Create kb_workflow/__init__.py to export main and process_command"
|
||||
- "Keep original module file names (bilibili_extractor.py, zhihu_extractor.py)"
|
||||
|
||||
patterns-established:
|
||||
- "Standard imports: from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor"
|
||||
- "No sys.path manipulation in platform adapters"
|
||||
|
||||
requirements-completed: [MET-01, MET-02, MET-03, MET-04, MET-05]
|
||||
|
||||
# Metrics
|
||||
duration: 7min
|
||||
completed: 2026-03-17
|
||||
---
|
||||
|
||||
# Phase 6 Plan 01: Metadata Module Consolidation Summary
|
||||
|
||||
**Unified metadata extraction modules into kb_workflow/metadata/ with standard Python package imports, eliminating sys.path manipulation**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 7 min
|
||||
- **Started:** 2026-03-16T15:58:02Z
|
||||
- **Completed:** 2026-03-17T00:05:00Z
|
||||
- **Tasks:** 4
|
||||
- **Files modified:** 10
|
||||
|
||||
## Accomplishments
|
||||
- Moved bilibili and zhihu metadata modules to kb_workflow/metadata/
|
||||
- Updated platform adapters to use standard imports
|
||||
- Deleted original kb-bilibili-metadata/ and kb-zhihu-metadata/ directories
|
||||
- Created kb_workflow package with proper __init__.py exports
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Create metadata directory and move bilibili modules** - `e2edce2` (feat)
|
||||
2. **Task 2: Move zhihu modules to metadata directory** - `a4a7454` (feat)
|
||||
3. **Task 3: Update platform adapters to use standard imports** - `98f8ec2` (refactor)
|
||||
4. **Task 4: Delete original metadata directories and update tests** - `7662833` (refactor)
|
||||
|
||||
## Files Created/Modified
|
||||
- `kb_workflow/__init__.py` - Package entry point, exports main and process_command
|
||||
- `kb_workflow/metadata/__init__.py` - Metadata package entry, exports extractors
|
||||
- `kb_workflow/metadata/bilibili_extractor.py` - Bilibili metadata extractor (moved)
|
||||
- `kb_workflow/metadata/bilibili_metadata.py` - Bilibili CLI (moved)
|
||||
- `kb_workflow/metadata/zhihu_extractor.py` - Zhihu Playwright extractor (moved)
|
||||
- `kb_workflow/metadata/zhihu_metadata.py` - Zhihu CLI (moved)
|
||||
- `kb_workflow/metadata/zhihu_article_crawler.py` - Zhihu article crawler (moved)
|
||||
- `kb_workflow/metadata/markdown_converter.py` - Markdown converter (moved)
|
||||
- `kb_pipeline/platforms/bilibili.py` - Updated imports, removed sys.path manipulation
|
||||
- `kb_pipeline/platforms/zhihu.py` - Updated imports, removed sys.path manipulation
|
||||
- `tests/test_cli/test_process_command.py` - Updated imports for new package structure
|
||||
|
||||
## Decisions Made
|
||||
- Renamed `kb-workflow` directory to `kb_workflow` for Python package compatibility (hyphens not allowed in package names)
|
||||
- Created `kb_workflow/__init__.py` to export `main` and `process_command` functions for backward compatibility with tests
|
||||
- Preserved original module file names to minimize changes and maintain clarity
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 3 - Blocking] Renamed kb-workflow to kb_workflow for package compatibility**
|
||||
- **Found during:** Task 1 (Create metadata directory)
|
||||
- **Issue:** Directory name `kb-workflow` contains hyphen, Python cannot import it as package
|
||||
- **Fix:** Used `git mv kb-workflow kb_workflow` to rename directory
|
||||
- **Files modified:** Entire kb-workflow directory renamed
|
||||
- **Verification:** `from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor` succeeds
|
||||
- **Committed in:** `e2edce2` (Task 1 commit)
|
||||
|
||||
**2. [Rule 3 - Blocking] Created kb_workflow/__init__.py for test compatibility**
|
||||
- **Found during:** Task 4 (Run tests)
|
||||
- **Issue:** Tests imported `kb_workflow` as module, needed `main` and `process_command` exports
|
||||
- **Fix:** Created `kb_workflow/__init__.py` with exports from `kb_workflow.py`
|
||||
- **Files modified:** `kb_workflow/__init__.py`
|
||||
- **Verification:** `from kb_workflow import main, process_command` succeeds
|
||||
- **Committed in:** `7662833` (Task 4 commit)
|
||||
|
||||
---
|
||||
|
||||
**Total deviations:** 2 auto-fixed (2 blocking)
|
||||
**Impact on plan:** Both fixes necessary for Python package conventions and test compatibility. No scope creep.
|
||||
|
||||
## Issues Encountered
|
||||
- Pre-existing test failures (14 tests) related to mock setup and test isolation, not caused by this refactoring
|
||||
- Tests pass when run individually, fail when run in full suite due to global state pollution between tests
|
||||
|
||||
## User Setup Required
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
- Metadata modules consolidated, ready for pipeline module consolidation (06-02)
|
||||
- Standard import pattern established for future module moves
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- All created files verified to exist
|
||||
- Original directories (kb-bilibili-metadata, kb-zhihu-metadata) confirmed deleted
|
||||
- All 4 task commits verified in git history
|
||||
|
||||
---
|
||||
*Phase: 06-code-consolidation*
|
||||
*Completed: 2026-03-17*
|
||||
Reference in New Issue
Block a user