256 lines
8.8 KiB
Markdown
256 lines
8.8 KiB
Markdown
---
|
|
phase: 06-code-consolidation
|
|
plan: 01
|
|
type: execute
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified:
|
|
- kb-workflow/metadata/__init__.py
|
|
- kb-workflow/metadata/bilibili_extractor.py
|
|
- kb-workflow/metadata/bilibili_metadata.py
|
|
- kb-workflow/metadata/zhihu_extractor.py
|
|
- kb-workflow/metadata/zhihu_metadata.py
|
|
- kb-workflow/metadata/zhihu_article_crawler.py
|
|
- kb-workflow/metadata/markdown_converter.py
|
|
- kb_pipeline/platforms/bilibili.py
|
|
- kb_pipeline/platforms/zhihu.py
|
|
autonomous: true
|
|
requirements:
|
|
- MET-01
|
|
- MET-02
|
|
- MET-03
|
|
- MET-04
|
|
- MET-05
|
|
|
|
must_haves:
|
|
truths:
|
|
- "kb_workflow.metadata.bilibili_extractor 可正常导入"
|
|
- "kb_workflow.metadata.zhihu_extractor 可正常导入"
|
|
- "kb_pipeline/platforms/ 模块可通过新路径导入 metadata 模块"
|
|
- "原 kb-bilibili-metadata 目录已删除"
|
|
- "原 kb-zhihu-metadata 目录已删除"
|
|
artifacts:
|
|
- path: "kb-workflow/metadata/__init__.py"
|
|
provides: "metadata 包入口，导出 BilibiliExtractor 和 ZhihuPlaywrightExtractor"
|
|
- path: "kb-workflow/metadata/bilibili_extractor.py"
|
|
provides: "B站元数据提取器"
|
|
- path: "kb-workflow/metadata/zhihu_extractor.py"
|
|
provides: "知乎内容提取器"
|
|
key_links:
|
|
- from: "kb_pipeline/platforms/bilibili.py"
|
|
to: "kb_workflow.metadata.bilibili_extractor"
|
|
via: "from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor"
|
|
- from: "kb_pipeline/platforms/zhihu.py"
|
|
to: "kb_workflow.metadata.zhihu_extractor"
|
|
via: "from kb_workflow.metadata.zhihu_extractor import ZhihuPlaywrightExtractor"
|
|
---
|
|
|
|
<objective>
|
|
将独立的元数据技能模块 (kb-bilibili-metadata 和 kb-zhihu-metadata) 整合进 kb-workflow/metadata/。
|
|
|
|
Purpose: 统一代码结构，消除 sys.path 动态操作，使用标准包导入。
|
|
Output: kb-workflow/metadata/ 目录包含所有元数据提取模块，platform adapters 使用标准导入。
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/06-code-consolidation/06-CONTEXT.md
|
|
@.planning/phases/06-code-consolidation/06-RESEARCH.md
|
|
|
|
<interfaces>
|
|
<!-- Key interfaces executor needs from codebase -->
|
|
|
|
From kb_pipeline/platforms/bilibili.py (需要修改的动态导入):
|
|
```python
|
|
# 旧代码 - 需要删除
|
|
_BILIBILI_METADATA_PATH = Path(__file__).parent.parent.parent / "kb-bilibili-metadata"
|
|
if _BILIBILI_METADATA_PATH.exists():
|
|
sys.path.insert(0, str(_BILIBILI_METADATA_PATH))
|
|
from bilibili_extractor import BilibiliExtractor
|
|
|
|
# 新代码 - 标准导入
|
|
from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor
|
|
```
|
|
|
|
From kb_pipeline/platforms/zhihu.py (需要修改的动态导入):
|
|
```python
|
|
# 旧代码 - 需要删除
|
|
_kb_zhihu_path = Path(__file__).parent.parent.parent / "kb-zhihu-metadata"
|
|
if str(_kb_zhihu_path) not in sys.path:
|
|
sys.path.insert(0, str(_kb_zhihu_path))
|
|
from zhihu_extractor import ZhihuPlaywrightExtractor
|
|
|
|
# 新代码 - 标准导入
|
|
from kb_workflow.metadata.zhihu_extractor import ZhihuPlaywrightExtractor
|
|
```
|
|
</interfaces>
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto">
|
|
<name>Task 1: 创建 metadata 目录并移动 bilibili 模块</name>
|
|
<files>
|
|
kb-workflow/metadata/__init__.py,
|
|
kb-workflow/metadata/bilibili_extractor.py,
|
|
kb-workflow/metadata/bilibili_metadata.py
|
|
</files>
|
|
<action>
|
|
1. 创建 kb-workflow/metadata/ 目录
|
|
2. 使用 git mv 移动文件（保留历史）:
|
|
- kb-bilibili-metadata/bilibili_extractor.py → kb-workflow/metadata/bilibili_extractor.py
|
|
- kb-bilibili-metadata/bilibili_metadata.py → kb-workflow/metadata/bilibili_metadata.py
|
|
3. 创建 kb-workflow/metadata/__init__.py，内容如下:
|
|
```python
|
|
"""KB Workflow Metadata Module
|
|
|
|
提供 B站和知乎的元数据提取功能
|
|
"""
|
|
|
|
from .bilibili_extractor import BilibiliExtractor
|
|
|
|
__all__ = [
|
|
"BilibiliExtractor",
|
|
]
|
|
```
|
|
4. 移动后不要修改 bilibili_extractor.py 内部代码（保持原样）
|
|
</action>
|
|
<verify>
|
|
<automated>python -c "from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor; print('bilibili OK')"</automated>
|
|
</verify>
|
|
<done>kb-workflow/metadata/ 目录存在，bilibili_extractor.py 可从新路径导入</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: 移动 zhihu 模块到 metadata 目录</name>
|
|
<files>
|
|
kb-workflow/metadata/zhihu_extractor.py,
|
|
kb-workflow/metadata/zhihu_metadata.py,
|
|
kb-workflow/metadata/zhihu_article_crawler.py,
|
|
kb-workflow/metadata/markdown_converter.py,
|
|
kb-workflow/metadata/__init__.py
|
|
</files>
|
|
<action>
|
|
1. 使用 git mv 移动文件:
|
|
- kb-zhihu-metadata/zhihu_extractor.py → kb-workflow/metadata/zhihu_extractor.py
|
|
- kb-zhihu-metadata/zhihu_metadata.py → kb-workflow/metadata/zhihu_metadata.py
|
|
- kb-zhihu-metadata/zhihu_article_crawler.py → kb-workflow/metadata/zhihu_article_crawler.py
|
|
- kb-zhihu-metadata/markdown_converter.py → kb-workflow/metadata/markdown_converter.py
|
|
2. 更新 kb-workflow/metadata/__init__.py:
|
|
```python
|
|
"""KB Workflow Metadata Module
|
|
|
|
提供 B站和知乎的元数据提取功能
|
|
"""
|
|
|
|
from .bilibili_extractor import BilibiliExtractor
|
|
from .zhihu_extractor import ZhihuPlaywrightExtractor
|
|
|
|
__all__ = [
|
|
"BilibiliExtractor",
|
|
"ZhihuPlaywrightExtractor",
|
|
]
|
|
```
|
|
3. 移动后不要修改内部代码（保持原样）
|
|
</action>
|
|
<verify>
|
|
<automated>python -c "from kb_workflow.metadata.zhihu_extractor import ZhihuPlaywrightExtractor; print('zhihu OK')"</automated>
|
|
</verify>
|
|
<done>zhihu 模块文件已移动到 kb-workflow/metadata/，可从新路径导入</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 3: 更新 platform adapters 导入路径</name>
|
|
<files>
|
|
kb_pipeline/platforms/bilibili.py,
|
|
kb_pipeline/platforms/zhihu.py
|
|
</files>
|
|
<action>
|
|
更新 kb_pipeline/platforms/bilibili.py:
|
|
1. 删除 sys.path 动态操作代码（第 33-36 行）:
|
|
```python
|
|
# 删除以下代码
|
|
_BILIBILI_METADATA_PATH = Path(__file__).parent.parent.parent / "kb-bilibili-metadata"
|
|
if _BILIBILI_METADATA_PATH.exists():
|
|
sys.path.insert(0, str(_BILIBILI_METADATA_PATH))
|
|
```
|
|
2. 将 `from bilibili_extractor import BilibiliExtractor` 改为 `from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor`
|
|
3. 删除 try/except 包装，直接导入（因为包已存在）
|
|
|
|
更新 kb_pipeline/platforms/zhihu.py:
|
|
1. 删除 sys.path 动态操作代码（第 18-20 行）:
|
|
```python
|
|
# 删除以下代码
|
|
_kb_zhihu_path = Path(__file__).parent.parent.parent / "kb-zhihu-metadata"
|
|
if str(_kb_zhihu_path) not in sys.path:
|
|
sys.path.insert(0, str(_kb_zhihu_path))
|
|
```
|
|
2. 将 `from zhihu_extractor import ZhihuPlaywrightExtractor` 改为 `from kb_workflow.metadata.zhihu_extractor import ZhihuPlaywrightExtractor`
|
|
3. 删除 try/except 包装中的 ImportError 处理逻辑，直接导入
|
|
</action>
|
|
<verify>
|
|
<automated>python -c "from kb_pipeline.platforms.bilibili import BilibiliAdapter; from kb_pipeline.platforms.zhihu import ZhihuAdapter; print('adapters OK')"</automated>
|
|
</verify>
|
|
<done>platform adapters 使用标准包导入，无 sys.path 操作</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 4: 删除原独立目录并验证</name>
|
|
<files>
|
|
(删除) kb-bilibili-metadata/,
|
|
(删除) kb-zhihu-metadata/
|
|
</files>
|
|
<action>
|
|
1. 运行测试验证整合成功:
|
|
```bash
|
|
pytest tests/test_platforms/test_bilibili.py tests/test_platforms/test_zhihu.py -v
|
|
```
|
|
2. 测试通过后，使用 git rm 删除原目录:
|
|
```bash
|
|
git rm -r kb-bilibili-metadata/ kb-zhihu-metadata/
|
|
```
|
|
3. 再次运行测试确认删除不影响功能:
|
|
```bash
|
|
pytest tests/ -v --tb=short -x
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>pytest tests/ -v --tb=short -x</automated>
|
|
</verify>
|
|
<done>原 kb-bilibili-metadata 和 kb-zhihu-metadata 目录已删除，所有测试通过</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
1. 导入验证:
|
|
- `from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor` 成功
|
|
- `from kb_workflow.metadata.zhihu_extractor import ZhihuPlaywrightExtractor` 成功
|
|
2. 平台适配器验证:
|
|
- `from kb_pipeline.platforms.bilibili import BilibiliAdapter` 成功
|
|
- `from kb_pipeline.platforms.zhihu import ZhihuAdapter` 成功
|
|
3. 测试验证:
|
|
- `pytest tests/ -v` 全部通过
|
|
4. 清理验证:
|
|
- `ls kb-bilibili-metadata/` 返回 "No such file or directory"
|
|
- `ls kb-zhihu-metadata/` 返回 "No such file or directory"
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- [ ] kb-workflow/metadata/ 目录包含 bilibili 和 zhihu 所有模块
|
|
- [ ] kb_pipeline/platforms/ 使用标准导入，无 sys.path 操作
|
|
- [ ] 所有测试通过 (pytest tests/ -v)
|
|
- [ ] 原 kb-bilibili-metadata 和 kb-zhihu-metadata 目录已删除
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/06-code-consolidation/06-01-SUMMARY.md`
|
|
</output>
|