134 lines
5.3 KiB
Markdown
134 lines
5.3 KiB
Markdown
---
|
|
phase: 06-code-consolidation
|
|
plan: 01
|
|
subsystem: code-consolidation
|
|
tags: [metadata, import-refactor, package-structure]
|
|
|
|
# Dependency graph
|
|
requires: []
|
|
provides:
|
|
- kb_workflow/metadata/ - unified metadata extraction modules
|
|
- Standard Python package imports (no sys.path manipulation)
|
|
affects: [06-02]
|
|
|
|
# Tech tracking
|
|
tech-stack:
|
|
added: []
|
|
patterns: [standard-package-imports, git-mv-for-history-preservation]
|
|
|
|
key-files:
|
|
created:
|
|
- kb_workflow/__init__.py
|
|
- kb_workflow/metadata/__init__.py
|
|
modified:
|
|
- kb_pipeline/platforms/bilibili.py
|
|
- kb_pipeline/platforms/zhihu.py
|
|
- tests/test_cli/test_process_command.py
|
|
|
|
key-decisions:
|
|
- "Rename kb-workflow to kb_workflow for Python package compatibility"
|
|
- "Create kb_workflow/__init__.py to export main and process_command"
|
|
- "Keep original module file names (bilibili_extractor.py, zhihu_extractor.py)"
|
|
|
|
patterns-established:
|
|
- "Standard imports: from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor"
|
|
- "No sys.path manipulation in platform adapters"
|
|
|
|
requirements-completed: [MET-01, MET-02, MET-03, MET-04, MET-05]
|
|
|
|
# Metrics
|
|
duration: 7min
|
|
completed: 2026-03-17
|
|
---
|
|
|
|
# Phase 6 Plan 01: Metadata Module Consolidation Summary
|
|
|
|
**Unified metadata extraction modules into kb_workflow/metadata/ with standard Python package imports, eliminating sys.path manipulation**
|
|
|
|
## Performance
|
|
|
|
- **Duration:** 7 min
|
|
- **Started:** 2026-03-16T15:58:02Z
|
|
- **Completed:** 2026-03-17T00:05:00Z
|
|
- **Tasks:** 4
|
|
- **Files modified:** 10
|
|
|
|
## Accomplishments
|
|
- Moved bilibili and zhihu metadata modules to kb_workflow/metadata/
|
|
- Updated platform adapters to use standard imports
|
|
- Deleted original kb-bilibili-metadata/ and kb-zhihu-metadata/ directories
|
|
- Created kb_workflow package with proper __init__.py exports
|
|
|
|
## Task Commits
|
|
|
|
Each task was committed atomically:
|
|
|
|
1. **Task 1: Create metadata directory and move bilibili modules** - `e2edce2` (feat)
|
|
2. **Task 2: Move zhihu modules to metadata directory** - `a4a7454` (feat)
|
|
3. **Task 3: Update platform adapters to use standard imports** - `98f8ec2` (refactor)
|
|
4. **Task 4: Delete original metadata directories and update tests** - `7662833` (refactor)
|
|
|
|
## Files Created/Modified
|
|
- `kb_workflow/__init__.py` - Package entry point, exports main and process_command
|
|
- `kb_workflow/metadata/__init__.py` - Metadata package entry, exports extractors
|
|
- `kb_workflow/metadata/bilibili_extractor.py` - Bilibili metadata extractor (moved)
|
|
- `kb_workflow/metadata/bilibili_metadata.py` - Bilibili CLI (moved)
|
|
- `kb_workflow/metadata/zhihu_extractor.py` - Zhihu Playwright extractor (moved)
|
|
- `kb_workflow/metadata/zhihu_metadata.py` - Zhihu CLI (moved)
|
|
- `kb_workflow/metadata/zhihu_article_crawler.py` - Zhihu article crawler (moved)
|
|
- `kb_workflow/metadata/markdown_converter.py` - Markdown converter (moved)
|
|
- `kb_pipeline/platforms/bilibili.py` - Updated imports, removed sys.path manipulation
|
|
- `kb_pipeline/platforms/zhihu.py` - Updated imports, removed sys.path manipulation
|
|
- `tests/test_cli/test_process_command.py` - Updated imports for new package structure
|
|
|
|
## Decisions Made
|
|
- Renamed `kb-workflow` directory to `kb_workflow` for Python package compatibility (hyphens not allowed in package names)
|
|
- Created `kb_workflow/__init__.py` to export `main` and `process_command` functions for backward compatibility with tests
|
|
- Preserved original module file names to minimize changes and maintain clarity
|
|
|
|
## Deviations from Plan
|
|
|
|
### Auto-fixed Issues
|
|
|
|
**1. [Rule 3 - Blocking] Renamed kb-workflow to kb_workflow for package compatibility**
|
|
- **Found during:** Task 1 (Create metadata directory)
|
|
- **Issue:** Directory name `kb-workflow` contains hyphen, Python cannot import it as package
|
|
- **Fix:** Used `git mv kb-workflow kb_workflow` to rename directory
|
|
- **Files modified:** Entire kb-workflow directory renamed
|
|
- **Verification:** `from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor` succeeds
|
|
- **Committed in:** `e2edce2` (Task 1 commit)
|
|
|
|
**2. [Rule 3 - Blocking] Created kb_workflow/__init__.py for test compatibility**
|
|
- **Found during:** Task 4 (Run tests)
|
|
- **Issue:** Tests imported `kb_workflow` as module, needed `main` and `process_command` exports
|
|
- **Fix:** Created `kb_workflow/__init__.py` with exports from `kb_workflow.py`
|
|
- **Files modified:** `kb_workflow/__init__.py`
|
|
- **Verification:** `from kb_workflow import main, process_command` succeeds
|
|
- **Committed in:** `7662833` (Task 4 commit)
|
|
|
|
---
|
|
|
|
**Total deviations:** 2 auto-fixed (2 blocking)
|
|
**Impact on plan:** Both fixes necessary for Python package conventions and test compatibility. No scope creep.
|
|
|
|
## Issues Encountered
|
|
- Pre-existing test failures (14 tests) related to mock setup and test isolation, not caused by this refactoring
|
|
- Tests pass when run individually, fail when run in full suite due to global state pollution between tests
|
|
|
|
## User Setup Required
|
|
None - no external service configuration required.
|
|
|
|
## Next Phase Readiness
|
|
- Metadata modules consolidated, ready for pipeline module consolidation (06-02)
|
|
- Standard import pattern established for future module moves
|
|
|
|
## Self-Check: PASSED
|
|
|
|
- All created files verified to exist
|
|
- Original directories (kb-bilibili-metadata, kb-zhihu-metadata) confirmed deleted
|
|
- All 4 task commits verified in git history
|
|
|
|
---
|
|
*Phase: 06-code-consolidation*
|
|
*Completed: 2026-03-17*
|