--- phase: 06-code-consolidation plan: 02 type: execute wave: 2 depends_on: - 06-01 files_modified: - kb-workflow/pipeline/__init__.py - kb-workflow/pipeline/pipeline.py - kb-workflow/pipeline/schemas.py - kb-workflow/pipeline/config.py - kb-workflow/pipeline/state.py - kb-workflow/pipeline/errors.py - kb-workflow/pipeline/retry.py - kb-workflow/pipeline/idempotency.py - kb-workflow/pipeline/link_parser.py - kb-workflow/pipeline/processors/__init__.py - kb-workflow/pipeline/processors/base.py - kb-workflow/pipeline/processors/parse.py - kb-workflow/pipeline/processors/download.py - kb-workflow/pipeline/processors/asr.py - kb-workflow/pipeline/processors/analyze.py - kb-workflow/pipeline/processors/archive.py - kb-workflow/pipeline/platforms/__init__.py - kb-workflow/pipeline/platforms/base.py - kb-workflow/pipeline/platforms/bilibili.py - kb-workflow/pipeline/platforms/zhihu.py - kb-workflow/kb_workflow.py - tests/test_pipeline/*.py - tests/test_processors/*.py - tests/test_platforms/*.py - tests/test_cli/*.py - tests/test_integration/*.py autonomous: true requirements: - PIP-01 - PIP-02 - PIP-03 - PIP-04 must_haves: truths: - "kb_workflow.pipeline 可正常导入" - "kb_workflow.pipeline.processors 包含所有处理器" - "kb_workflow.pipeline.platforms 包含所有平台适配器" - "CLI 入口 kb_workflow.py 可正常执行" - "所有测试通过" - "原 kb_pipeline 目录已删除" artifacts: - path: "kb-workflow/pipeline/__init__.py" provides: "pipeline 包入口,导出核心类和函数" - path: "kb-workflow/pipeline/pipeline.py" provides: "Pipeline 编排引擎" - path: "kb-workflow/pipeline/schemas.py" provides: "Pydantic Schema 定义" - path: "kb-workflow/pipeline/processors/" provides: "所有处理器实现" - path: "kb-workflow/pipeline/platforms/" provides: "所有平台适配器" - path: "kb-workflow/kb_workflow.py" provides: "CLI 入口" key_links: - from: "kb-workflow/kb_workflow.py" to: "kb_workflow.pipeline" via: "from kb_workflow.pipeline.xxx import ..." - from: "tests/" to: "kb_workflow.pipeline" via: "from kb_workflow.pipeline.xxx import ..." - from: "kb-workflow/pipeline/processors/base.py" to: "kb_workflow.pipeline.config" via: "from kb_workflow.pipeline.config import get_settings" - from: "kb-workflow/pipeline/platforms/bilibili.py" to: "kb_workflow.pipeline.errors" via: "from kb_workflow.pipeline.errors import NetworkError" --- 将 kb_pipeline/ 模块合并进 kb-workflow/pipeline/,更新所有导入路径,删除原目录。 Purpose: 完成代码整合,所有代码统一在 kb_workflow 包下。 Output: kb-workflow/pipeline/ 目录包含所有 pipeline 模块,CLI 和测试使用新导入路径。 @/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md @/home/starmen1265/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/06-code-consolidation/06-CONTEXT.md @.planning/phases/06-code-consolidation/06-RESEARCH.md @.planning/phases/06-code-consolidation/06-01-SUMMARY.md From kb-workflow/kb_workflow.py (需要修改的导入): ```python # 旧代码 from kb_pipeline.config import get_settings from kb_pipeline.pipeline import Pipeline from kb_pipeline.processors import ( ASRProcessor, AnalyzeProcessor, ArchiveProcessor, DownloadProcessor, ParseProcessor ) from kb_pipeline.schemas import PipelineConfig # 新代码 from kb_workflow.pipeline.config import get_settings from kb_workflow.pipeline.pipeline import Pipeline from kb_workflow.pipeline.processors import ( ASRProcessor, AnalyzeProcessor, ArchiveProcessor, DownloadProcessor, ParseProcessor ) from kb_workflow.pipeline.schemas import PipelineConfig ``` From kb_pipeline/__init__.py (需要移动后更新导入): ```python # 所有 from .xxx 改为 from kb_workflow.pipeline.xxx from kb_workflow.pipeline.schemas import (...) from kb_workflow.pipeline.state import WorkflowStateManager from kb_workflow.pipeline.errors import (...) from kb_workflow.pipeline.config import KBSettings, get_settings from kb_workflow.pipeline.retry import retry_on_network_error from kb_workflow.pipeline.link_parser import LinkParser, ParsedLink from kb_workflow.pipeline.idempotency import IdempotencyChecker from kb_workflow.pipeline.platforms import PlatformAdapter, BilibiliAdapter, ZhihuAdapter ``` From kb_pipeline/processors/base.py (内部导入需要更新): ```python # 旧 from kb_pipeline.config import get_settings from kb_pipeline.schemas import StepInput, StepOutput # 新 from kb_workflow.pipeline.config import get_settings from kb_workflow.pipeline.schemas import StepInput, StepOutput ``` From kb_pipeline/platforms/bilibili.py (已更新的 metadata 导入): ```python # 在 06-01 中已更新为: from kb_workflow.metadata.bilibili_extractor import BilibiliExtractor # 还需要更新内部导入: # 旧 from .base import PlatformAdapter from ..errors import NetworkError, AuthError from ..retry import retry_on_network_error from ..state import WorkflowStateManager # 新 from kb_workflow.pipeline.platforms.base import PlatformAdapter from kb_workflow.pipeline.errors import NetworkError, AuthError from kb_workflow.pipeline.retry import retry_on_network_error from kb_workflow.pipeline.state import WorkflowStateManager ``` Task 1: 创建 pipeline 目录并移动核心文件 kb-workflow/pipeline/__init__.py, kb-workflow/pipeline/pipeline.py, kb-workflow/pipeline/schemas.py, kb-workflow/pipeline/config.py, kb-workflow/pipeline/state.py, kb-workflow/pipeline/errors.py, kb-workflow/pipeline/retry.py, kb-workflow/pipeline/idempotency.py, kb-workflow/pipeline/link_parser.py 1. 创建 kb-workflow/pipeline/ 目录 2. 使用 git mv 移动核心文件: ```bash git mv kb_pipeline/pipeline.py kb-workflow/pipeline/ git mv kb_pipeline/schemas.py kb-workflow/pipeline/ git mv kb_pipeline/config.py kb-workflow/pipeline/ git mv kb_pipeline/state.py kb-workflow/pipeline/ git mv kb_pipeline/errors.py kb-workflow/pipeline/ git mv kb_pipeline/retry.py kb-workflow/pipeline/ git mv kb_pipeline/idempotency.py kb-workflow/pipeline/ git mv kb_pipeline/link_parser.py kb-workflow/pipeline/ ``` 3. 创建 kb-workflow/pipeline/__init__.py(基于原 kb_pipeline/__init__.py,更新导入路径): ```python """KB Pipeline - 工作流管道基础设施 提供内容处理工作流的核心 Schema 和工具。 """ from kb_workflow.pipeline.schemas import ( StepInput, StepOutput, StepResult, StepStatus, StepRecord, WorkflowState, StepDefinition, ) from kb_workflow.pipeline.state import WorkflowStateManager from kb_workflow.pipeline.errors import ( KBError, NetworkError, ValidationError, SystemError, AuthError, ErrorType, ) from kb_workflow.pipeline.config import KBSettings, get_settings from kb_workflow.pipeline.retry import retry_on_network_error from kb_workflow.pipeline.link_parser import LinkParser, ParsedLink from kb_workflow.pipeline.idempotency import IdempotencyChecker from kb_workflow.pipeline.platforms import PlatformAdapter, BilibiliAdapter, ZhihuAdapter __all__ = [ # Schemas "StepInput", "StepOutput", "StepResult", "StepStatus", "StepRecord", "WorkflowState", "StepDefinition", # State "WorkflowStateManager", # Errors "KBError", "NetworkError", "ValidationError", "SystemError", "AuthError", "ErrorType", # Config "KBSettings", "get_settings", # Retry "retry_on_network_error", # Link Parser "LinkParser", "ParsedLink", # Idempotency "IdempotencyChecker", # Platform Adapters "PlatformAdapter", "BilibiliAdapter", "ZhihuAdapter", ] __version__ = "0.1.0" ``` 4. 更新核心文件内部导入(使用绝对导入): - pipeline.py: 所有 `from kb_pipeline.xxx` 改为 `from kb_workflow.pipeline.xxx` - schemas.py: 检查是否有内部导入需要更新 - config.py: 检查是否有内部导入需要更新 - state.py: 检查是否有内部导入需要更新 - errors.py: 通常无外部导入 - retry.py: 通常无外部导入 - idempotency.py: 通常无外部导入 - link_parser.py: 通常无外部导入 python -c "from kb_workflow.pipeline import Pipeline, get_settings, WorkflowStateManager; print('core OK')" pipeline 核心文件已移动到 kb-workflow/pipeline/,核心类可导入 Task 2: 移动 processors 和 platforms 子目录 kb-workflow/pipeline/processors/__init__.py, kb-workflow/pipeline/processors/base.py, kb-workflow/pipeline/processors/parse.py, kb-workflow/pipeline/processors/download.py, kb-workflow/pipeline/processors/asr.py, kb-workflow/pipeline/processors/analyze.py, kb-workflow/pipeline/processors/archive.py, kb-workflow/pipeline/platforms/__init__.py, kb-workflow/pipeline/platforms/base.py, kb-workflow/pipeline/platforms/bilibili.py, kb-workflow/pipeline/platforms/zhihu.py 1. 创建子目录并移动文件: ```bash mkdir -p kb-workflow/pipeline/processors mkdir -p kb-workflow/pipeline/platforms git mv kb_pipeline/processors/*.py kb-workflow/pipeline/processors/ git mv kb_pipeline/platforms/*.py kb-workflow/pipeline/platforms/ ``` 2. 更新 processors 内部导入(改为绝对导入): 在以下文件中将 `from kb_pipeline.xxx` 改为 `from kb_workflow.pipeline.xxx`: - base.py: `from kb_pipeline.config` → `from kb_workflow.pipeline.config` - base.py: `from kb_pipeline.schemas` → `from kb_workflow.pipeline.schemas` - parse.py, download.py, asr.py, analyze.py, archive.py: 检查并更新所有 kb_pipeline 导入 - __init__.py: 更新所有 from .xxx 或 from ..xxx 为绝对导入 3. 更新 platforms 内部导入: - bilibili.py: 已在 06-01 更新 metadata 导入,现在更新: - `from .base` → `from kb_workflow.pipeline.platforms.base` - `from ..errors` → `from kb_workflow.pipeline.errors` - `from ..retry` → `from kb_workflow.pipeline.retry` - `from ..state` → `from kb_workflow.pipeline.state` - `from ..config` → `from kb_workflow.pipeline.config` - zhihu.py: 同样更新所有相对导入为绝对导入 - base.py: 检查并更新导入 - __init__.py: 更新导出 4. 更新后的 processors/__init__.py: ```python """Pipeline Processors 提供所有管道处理器实现。 """ from kb_workflow.pipeline.processors.base import BaseProcessor from kb_workflow.pipeline.processors.parse import ParseProcessor from kb_workflow.pipeline.processors.download import DownloadProcessor from kb_workflow.pipeline.processors.asr import ASRProcessor from kb_workflow.pipeline.processors.analyze import AnalyzeProcessor from kb_workflow.pipeline.processors.archive import ArchiveProcessor __all__ = [ "BaseProcessor", "ParseProcessor", "DownloadProcessor", "ASRProcessor", "AnalyzeProcessor", "ArchiveProcessor", ] ``` 5. 更新后的 platforms/__init__.py: ```python """Platform Adapters 提供平台适配器接口和实现。 """ from kb_workflow.pipeline.platforms.base import PlatformAdapter from kb_workflow.pipeline.platforms.bilibili import BilibiliAdapter from kb_workflow.pipeline.platforms.zhihu import ZhihuAdapter __all__ = [ "PlatformAdapter", "BilibiliAdapter", "ZhihuAdapter", ] ``` python -c "from kb_workflow.pipeline.processors import ParseProcessor, DownloadProcessor, ASRProcessor, AnalyzeProcessor, ArchiveProcessor; from kb_workflow.pipeline.platforms import BilibiliAdapter, ZhihuAdapter; print('processors and platforms OK')" processors 和 platforms 已移动,所有处理器和适配器可导入 Task 3: 更新 CLI 入口和测试文件导入 kb-workflow/kb_workflow.py, tests/test_pipeline/*.py, tests/test_processors/*.py, tests/test_platforms/*.py, tests/test_cli/*.py, tests/test_integration/*.py 1. 更新 kb-workflow/kb_workflow.py: - 删除 sys.path.insert 操作(第 41-43 行) - 更新所有导入: ```python from kb_workflow.pipeline.config import get_settings from kb_workflow.pipeline.pipeline import Pipeline from kb_workflow.pipeline.processors import ( ASRProcessor, AnalyzeProcessor, ArchiveProcessor, DownloadProcessor, ParseProcessor ) from kb_workflow.pipeline.schemas import PipelineConfig ``` 2. 使用 grep 找出所有需要更新的测试文件: ```bash grep -rn "from kb_pipeline" tests/ --include="*.py" ``` 3. 批量更新测试文件导入(使用 sed 或手动): - `from kb_pipeline.xxx` → `from kb_workflow.pipeline.xxx` - 例如: `from kb_pipeline.schemas` → `from kb_workflow.pipeline.schemas` - 例如: `from kb_pipeline.processors` → `from kb_workflow.pipeline.processors` - 例如: `from kb_pipeline.platforms` → `from kb_workflow.pipeline.platforms` 4. 更新 tests/test_zhihu_metadata/ 中的导入(如果有) cd /home/starmen1265/work/kb-skills/kb-workflow && python kb_workflow.py --help CLI 入口可执行,测试文件导入路径已更新 Task 4: 删除原 kb_pipeline 目录并运行全量测试 (删除) kb_pipeline/ 1. 运行全量测试验证整合成功: ```bash pytest tests/ -v --tb=short -x ``` 2. 测试通过后,删除原目录: ```bash git rm -r kb_pipeline/ ``` 3. 再次运行测试确认删除不影响功能: ```bash pytest tests/ -v ``` 4. 验证 CLI 功能: ```bash cd kb-workflow && python kb_workflow.py --help ``` pytest tests/ -v --tb=short 原 kb_pipeline 目录已删除,所有测试通过,CLI 正常工作 1. 核心导入验证: - `from kb_workflow.pipeline import Pipeline, get_settings` 成功 - `from kb_workflow.pipeline.processors import ParseProcessor` 成功 - `from kb_workflow.pipeline.platforms import BilibiliAdapter` 成功 2. CLI 验证: - `python kb-workflow/kb_workflow.py --help` 输出帮助信息 3. 测试验证: - `pytest tests/ -v` 全部通过 4. 清理验证: - `ls kb_pipeline/` 返回 "No such file or directory" - [ ] kb-workflow/pipeline/ 目录包含所有 pipeline 模块 - [ ] kb-workflow/kb_workflow.py 使用新导入路径 - [ ] 所有测试文件使用新导入路径 - [ ] 所有测试通过 (pytest tests/ -v) - [ ] CLI 可执行 (python kb_workflow.py --help) - [ ] 原 kb_pipeline 目录已删除 After completion, create `.planning/phases/06-code-consolidation/06-02-SUMMARY.md`