starmen1265/kb-skills
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(06-01): complete metadata consolidation plan

Browse Source
This commit is contained in:
starmen1265
2026-03-17 00:08:09 +08:00
parent 766283366e
commit 65fdf4646a
3 changed files with 173 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/REQUIREMENTS.md
+10 -10
View File
@@ -27,11 +27,11 @@
### 代码整合 - 元数据 (Metadata Integration) ### 代码整合 - 元数据 (Metadata Integration)
- [ ] **MET-01**: kb-bilibili-metadata 整合进 kb-workflow/metadata/ - [x] **MET-01**: kb-bilibili-metadata 整合进 kb-workflow/metadata/
- [ ] **MET-02**: kb-zhihu-metadata 整合进 kb-workflow/metadata/ - [x] **MET-02**: kb-zhihu-metadata 整合进 kb-workflow/metadata/
- [ ] **MET-03**: 删除独立的 kb-bilibili-metadata 目录 - [x] **MET-03**: 删除独立的 kb-bilibili-metadata 目录
- [ ] **MET-04**: 删除独立的 kb-zhihu-metadata 目录 - [x] **MET-04**: 删除独立的 kb-zhihu-metadata 目录
- [ ] **MET-05**: 更新所有导入路径指向新位置 - [x] **MET-05**: 更新所有导入路径指向新位置
### 代码整合 - Pipeline (Pipeline Integration) ### 代码整合 - Pipeline (Pipeline Integration)
@@ -80,11 +80,11 @@ Deferred to future release.
| OUT-02 | Phase 5 | Complete | | OUT-02 | Phase 5 | Complete |
| OUT-03 | Phase 5 | Complete | | OUT-03 | Phase 5 | Complete |
| OUT-04 | Phase 5 | Complete | | OUT-04 | Phase 5 | Complete |
| MET-01 | Phase 6 | Pending | | MET-01 | Phase 6 | Complete |
| MET-02 | Phase 6 | Pending | | MET-02 | Phase 6 | Complete |
| MET-03 | Phase 6 | Pending | | MET-03 | Phase 6 | Complete |
| MET-04 | Phase 6 | Pending | | MET-04 | Phase 6 | Complete |
| MET-05 | Phase 6 | Pending | | MET-05 | Phase 6 | Complete |
| PIP-01 | Phase 6 | Pending | | PIP-01 | Phase 6 | Pending |
| PIP-02 | Phase 6 | Pending | | PIP-02 | Phase 6 | Pending |
| PIP-03 | Phase 6 | Pending | | PIP-03 | Phase 6 | Pending |
.planning/STATE.md
+30 -31
View File
@@ -2,16 +2,16 @@
gsd_state_version: 1.0 gsd_state_version: 1.0
milestone: v1.1 milestone: v1.1
milestone_name: Architecture Refactor milestone_name: Architecture Refactor
status: completed status: in_progress
stopped_at: Phase 6 context gathered stopped_at: 06-01 Metadata consolidation completed
last_updated: "2026-03-16T15:42:35.963Z" last_updated: "2026-03-17T00:10:00.000Z"
last_activity: 2026-03-16 — 05-03 Archive testing completed last_activity: 2026-03-17 — 06-01 Metadata consolidation completed
progress: progress:
total_phases: 6 total_phases: 6
completed_phases: 5 completed_phases: 5
total_plans: 15 total_plans: 17
completed_plans: 15 completed_plans: 16
percent: 10 percent: 94
--- ---
# Project State # Project State
@@ -21,49 +21,48 @@ progress:
See: .planning/PROJECT.md (updated 2026-03-16) See: .planning/PROJECT.md (updated 2026-03-16)
**Core value:** 可靠、可维护的处理管道 - 每个步骤独立、可验证、可恢复 **Core value:** 可靠、可维护的处理管道 - 每个步骤独立、可验证、可恢复
**Current focus:** v1.1 Architecture Refactor — Phase 5 归档与目录优化 COMPLETE **Current focus:** v1.1 Architecture Refactor — Phase 6 代码整合 IN PROGRESS
## Current Position ## Current Position
Phase: 5 of 6 (归档与目录优化) - COMPLETE Phase: 6 of 6 (代码整合) - IN PROGRESS
Plan: 3 of 3 in current phase - COMPLETE Plan: 1 of 2 in current phase - COMPLETE
Status: Phase 5 Complete Status: 06-01 Complete
Last activity: 2026-03-16 — 05-03 Archive testing completed Last activity: 2026-03-17 — 06-01 Metadata consolidation completed
Progress: [██░░░░░░░░] 10% (v1.1 milestone) Progress: [█████████░] 94% (v1.1 milestone)
## Performance Metrics ## Performance Metrics
**Velocity:** **Velocity:**
- Total plans completed: 3 (v1.1) - Total plans completed: 4 (v1.1)
- Average duration: 3 min - Average duration: 4 min
- Total execution time: 9 min - Total execution time: 16 min
**By Phase:** **By Phase:**
| Phase | Plans | Total | Avg/Plan | | Phase | Plans | Total | Avg/Plan |
|-------|-------|-------|----------| |-------|-------|-------|----------|
| 5. 归档与目录优化 | 3/3 | 9 min | 3 min | | 5. 归档与目录优化 | 3/3 | 9 min | 3 min |
| 6. 代码整合 | 0/2 | - | - | | 6. 代码整合 | 1/2 | 7 min | 7 min |
**Recent Trend:** **Recent Trend:**
- Last 5 plans: 3 min - Last 5 plans: 5 min
- Trend: Stable - Trend: Stable
*Updated after each plan completion* *Updated after each plan completion*
| Phase 05 P03 | 6min | 3 tasks | 4 files | | Phase 06 P01 | 7min | 4 tasks | 10 files |
| Phase 05 P03 | 6min | 3 tasks | 4 files |
## Accumulated Context ## Accumulated Context
### Decisions ### Decisions
Key decisions from v1.0 (preserved): Key decisions from v1.0 (preserved):
- ✓ 管道模型 — BaseProcessor + Pipeline 架构 - 管道模型 — BaseProcessor + Pipeline 架构
- ✓ 步骤级状态 — WorkflowStateManager - 步骤级状态 — WorkflowStateManager
- ✓ Schema 验证 — Pydantic v2 - Schema 验证 — Pydantic v2
- ✓ 顺序执行 — 简单性和可调试性 - 顺序执行 — 简单性和可调试性
- ✓ CLI subparsers — 可扩展命令结构 - CLI subparsers — 可扩展命令结构
v1.1 decisions: v1.1 decisions:
- 中间目录统一到 `~/work/intermediate/analysis/` - 中间目录统一到 `~/work/intermediate/analysis/`
@@ -72,9 +71,9 @@ v1.1 decisions:
- [Phase 05]: Archive step is always last in pipeline, only md files archived - [Phase 05]: Archive step is always last in pipeline, only md files archived
- [Phase 05]: Duplicate detection uses glob pattern *_{content_id}_analysis.md with user overwrite confirmation - [Phase 05]: Duplicate detection uses glob pattern *_{content_id}_analysis.md with user overwrite confirmation
- [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking - [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking
- [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level
- [Phase 05]: Mock _ask_overwrite in tests to avoid stdin blocking
- [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level for tests - [Phase 05]: Patch get_settings at kb_pipeline.processors.base module level for tests
- [Phase 06]: Rename kb-workflow to kb_workflow for Python package compatibility
- [Phase 06]: Standard imports from kb_workflow.metadata instead of sys.path manipulation
### Blockers/Concerns ### Blockers/Concerns
@@ -82,8 +81,8 @@ None.
## Session Continuity ## Session Continuity
Last session: 2026-03-16T15:42:35.962Z Last session: 2026-03-17T00:10:00.000Z
Stopped at: Phase 6 context gathered Stopped at: 06-01 Metadata consolidation completed
Resume file: .planning/phases/06-code-consolidation/06-CONTEXT.md Resume file: .planning/phases/06-code-consolidation/06-02-PLAN.md
**Next step:** Execute 06-01-PLAN.md (Phase 6: Code Consolidation) **Next step:** Execute 06-02-PLAN.md (Phase 6: Pipeline Module Consolidation)
.planning/phases/06-code-consolidation/06-01-SUMMARY.md
+133
View File
@@ -0,0 +1,133 @@
---
phase: 06-code-consolidation
plan: 01
subsystem: code-consolidation
tags: [metadata, import-refactor, package-structure]
# Dependency graph
requires: []
provides:
- kb_workflow/metadata/ - unified metadata extraction modules
- Standard Python package imports (no sys.path manipulation)
affects: [06-02]
# Tech tracking
tech-stack:
added: []
patterns: [standard-package-imports, git-mv-for-history-preservation]
key-files:
created:
- kb_workflow/__init__.py
- kb_workflow/metadata/__init__.py
modified:
- kb_pipeline/platforms/bilibili.py
- kb_pipeline/platforms/zhihu.py
- tests/test_cli/test_process_command.py
key-decisions:
- "Rename kb-workflow to kb_workflow for Python package compatibility"
- "Create kb_workflow/__init__.py to export main and process_command"
- "Keep original module file names (bilibili_extractor.py, zhihu_extractor.py)"
patterns-established:
- "Standard imports: from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor"
- "No sys.path manipulation in platform adapters"
requirements-completed: [MET-01, MET-02, MET-03, MET-04, MET-05]
# Metrics
duration: 7min
completed: 2026-03-17
---
# Phase 6 Plan 01: Metadata Module Consolidation Summary
**Unified metadata extraction modules into kb_workflow/metadata/ with standard Python package imports, eliminating sys.path manipulation**
## Performance
- **Duration:** 7 min
- **Started:** 2026-03-16T15:58:02Z
- **Completed:** 2026-03-17T00:05:00Z
- **Tasks:** 4
- **Files modified:** 10
## Accomplishments
- Moved bilibili and zhihu metadata modules to kb_workflow/metadata/
- Updated platform adapters to use standard imports
- Deleted original kb-bilibili-metadata/ and kb-zhihu-metadata/ directories
- Created kb_workflow package with proper __init__.py exports
## Task Commits
Each task was committed atomically:
1. **Task 1: Create metadata directory and move bilibili modules** - `e2edce2` (feat)
2. **Task 2: Move zhihu modules to metadata directory** - `a4a7454` (feat)
3. **Task 3: Update platform adapters to use standard imports** - `98f8ec2` (refactor)
4. **Task 4: Delete original metadata directories and update tests** - `7662833` (refactor)
## Files Created/Modified
- `kb_workflow/__init__.py` - Package entry point, exports main and process_command
- `kb_workflow/metadata/__init__.py` - Metadata package entry, exports extractors
- `kb_workflow/metadata/bilibili_extractor.py` - Bilibili metadata extractor (moved)
- `kb_workflow/metadata/bilibili_metadata.py` - Bilibili CLI (moved)
- `kb_workflow/metadata/zhihu_extractor.py` - Zhihu Playwright extractor (moved)
- `kb_workflow/metadata/zhihu_metadata.py` - Zhihu CLI (moved)
- `kb_workflow/metadata/zhihu_article_crawler.py` - Zhihu article crawler (moved)
- `kb_workflow/metadata/markdown_converter.py` - Markdown converter (moved)
- `kb_pipeline/platforms/bilibili.py` - Updated imports, removed sys.path manipulation
- `kb_pipeline/platforms/zhihu.py` - Updated imports, removed sys.path manipulation
- `tests/test_cli/test_process_command.py` - Updated imports for new package structure
## Decisions Made
- Renamed `kb-workflow` directory to `kb_workflow` for Python package compatibility (hyphens not allowed in package names)
- Created `kb_workflow/__init__.py` to export `main` and `process_command` functions for backward compatibility with tests
- Preserved original module file names to minimize changes and maintain clarity
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 3 - Blocking] Renamed kb-workflow to kb_workflow for package compatibility**
- **Found during:** Task 1 (Create metadata directory)
- **Issue:** Directory name `kb-workflow` contains hyphen, Python cannot import it as package
- **Fix:** Used `git mv kb-workflow kb_workflow` to rename directory
- **Files modified:** Entire kb-workflow directory renamed
- **Verification:** `from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor` succeeds
- **Committed in:** `e2edce2` (Task 1 commit)
**2. [Rule 3 - Blocking] Created kb_workflow/__init__.py for test compatibility**
- **Found during:** Task 4 (Run tests)
- **Issue:** Tests imported `kb_workflow` as module, needed `main` and `process_command` exports
- **Fix:** Created `kb_workflow/__init__.py` with exports from `kb_workflow.py`
- **Files modified:** `kb_workflow/__init__.py`
- **Verification:** `from kb_workflow import main, process_command` succeeds
- **Committed in:** `7662833` (Task 4 commit)
---
**Total deviations:** 2 auto-fixed (2 blocking)
**Impact on plan:** Both fixes necessary for Python package conventions and test compatibility. No scope creep.
## Issues Encountered
- Pre-existing test failures (14 tests) related to mock setup and test isolation, not caused by this refactoring
- Tests pass when run individually, fail when run in full suite due to global state pollution between tests
## User Setup Required
None - no external service configuration required.
## Next Phase Readiness
- Metadata modules consolidated, ready for pipeline module consolidation (06-02)
- Standard import pattern established for future module moves
## Self-Check: PASSED
- All created files verified to exist
- Original directories (kb-bilibili-metadata, kb-zhihu-metadata) confirmed deleted
- All 4 task commits verified in git history
---
*Phase: 06-code-consolidation*
*Completed: 2026-03-17*