docs(04-analysis-orchestration): create phase plans

This commit is contained in:
2026-03-13 22:50:28 +08:00
parent debd53961a
commit 4b4ed9cab6
6 changed files with 1730 additions and 0 deletions
@@ -0,0 +1,288 @@
---
phase: 04-analysis-orchestration
plan: 05
type: execute
wave: 5
depends_on: [04-04]
files_modified:
- README.md
- tests/test_integration/test_e2e_workflow.py
autonomous: false
requirements: [MNT-01]
user_setup: []
must_haves:
truths:
- "End-to-end workflow completes successfully"
- "Analysis report is saved to content directory"
- "workflow_state.json correctly records all steps"
artifacts:
- path: "README.md"
provides: "Project documentation"
contains: "## Installation"
- path: "tests/test_integration/test_e2e_workflow.py"
provides: "End-to-end integration test"
contains: "test_e2e"
key_links:
- from: "kb_pipeline/pipeline.py"
to: "kb_pipeline/processors"
via: "import processors"
pattern: "from.*processor import"
- from: "kb-workflow/kb_workflow.py"
to: "kb_pipeline"
via: "import pipeline"
pattern: "from.*pipeline import"
---
<objective>
Create end-to-end integration test and documentation ( project overview, installation, and and usage.
Purpose: Verify the complete workflow works and provide clear project documentation for maintainability.
Output: README.md with project overview, tests/test_integration/test_e2e_workflow.py
</objective>
<execution_context>
@/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md
@/home/starmen1265/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/04-analysis-orchestration/04-CONTEXT.md
@.planning/phases/04-analysis-orchestration/04-RESEARCH.md
@.planning/phases/04-analysis-orchestration/04-04-SUMMARY.md
<interfaces>
<!-- Key types and contracts from existing code. Executor should use these directly. -->
From kb_pipeline/pipeline.py:
```python
class Pipeline:
"""Pipeline orchestration engine"""
def __init__(self, ...): # From 04-01
...
```
From kb_pipeline/processors/base.py:
```python
class BaseProcessor(ABC):
STEP_NAME: str
async def process(self, input_data: StepInput) -> StepOutput
```
from kb_pipeline/processors/parse.py:
```python
class ParseProcessor(BaseProcessor):
STEP_NAME = "parse"
async def process(self, input_data: ParseInput) -> ParseOutput
```
from kb_pipeline/processors/download.py:
```python
class DownloadProcessor(BaseProcessor):
STEP_NAME = "download"
async def process(self, input_data: DownloadInput) -> DownloadOutput
```
from kb_pipeline/processors/asr.py:
```python
class ASRProcessor:
STEP_NAME = "asr"
async def process(self, input_data: ASRInput) -> ASROutput
```
from kb_pipeline/processors/analyze.py
```python
class AnalyzeProcessor(BaseProcessor):
STEP_NAME = "analyze"
async def process(self, input_data: AnalyzeInput) -> AnalyOutput
```
</interfaces>
</context>
<tasks>
<task type="auto" tdd="true">
<name>Task 1: Create README.md</name>
<files>README.md</files>
<behavior>
- Test 1: README.md exists and contains project overview
- Test 2: README.md contains installation instructions
</behavior>
<action>
Create `README.md` in project root directory with the following sections:
**## kb-skills**
知识库工作流工具,从链接到分析报告的端到端自动化处理系统。
### 格 Features
- 统一 Processor 模式: Parse -> Download -> ASR -> Analyze
- 状态管理: workflow_state.json
- 配置管理: 通过 ~/.kb_env
- 输出: Markdown 格式分析报告
- 幂等性保证: 重复执行不会产生重复文件
- 断点续传: 中断后可继续
- 模块化设计: 新人能在30分钟内理解代码结构
- 清晰的项目结构
- 统一的 CLI 入
entry
### Installation
```bash
# 安装依赖
pip install kb-pipeline
```
验证安装:
```bash
# 测试
pytest tests/test_pipeline/test_schemas.py tests/test_pipeline/test_state.py tests/test_pipeline/test_config.py tests/test_pipeline/test_errors.py tests/test_pipeline/test_retry.py
```
</action>
<verify>
<automated>python -c "import os; content = open('README.md', 'r', encoding='utf-8'); print(content[:100]) > 0"
</verify>
<done>README.md created with project overview, installation, and, usage sections, docstrings</done>
</task>
<task type="auto" tdd="true">
<name>Task 2: Create end-to-end integration test</name>
<files>tests/test_integration/test_e2e_workflow.py</files>
<behavior>
- Test 1: Full workflow executes from URL to analysis
- Test 2: Workflow handles errors gracefully
- Test 3: State is persisted correctly
- Test 4: Workflow is resumable
</behavior>
<action>
Create `tests/test_integration/test_e2e_workflow.py` with comprehensive end-to-end tests.
**Test structure:**
```python
import pytest
import pytest_asyncio
from unittest.mock import AsyncMock, patch, MagicMock
from pathlib import Path
from typing import Dict, Any
from kb_pipeline.pipeline import Pipeline
from kb_pipeline.config import get_settings
from kb_pipeline.processors import ParseProcessor, DownloadProcessor, ASRProcessor, AnalyzeProcessor
@pytest.fixture
def temp_content_dir(tmp_path):
"""Create temp content directory"""
content_dir.mkdir(parents=True, exist_ok=True)
yield content_dir
# Create test workflow state
state_path = content_dir / "workflow_state.json"
state = WorkflowState(
content_id="test123",
platform="bilibili",
source_url="https://test.url"
)
state_manager = WorkflowStateManager(content_dir)
yield state_manager
# Create real pipeline
settings = get_settings()
pipeline = Pipeline(
processors=[
ParseProcessor(state_manager),
DownloadProcessor(state_manager),
ASRProcessor(state_manager),
AnalyzeProcessor(state_manager),
],
output_dir=content_dir,
config=PipelineConfig(
steps=["parse", "download", "asr", "analyze"],
)
)
yield pipeline
# Mock workflow run
result = await pipeline.run("https://test.url")
# Verify success
assert result["parse"]["success"]
assert result["download"]["success"]
assert result["asr"]["success"] or result["asr"]["skipped"] # ASR not applicable for text content
assert result["analyze"]["success"]
# Verify state was saved
state = state_manager.load()
assert state is not None
assert state.is_step_completed("parse")
assert state.is_step_completed("download")
if content_type in ["video", "zvideo"]:
assert state.is_step_completed("asr")
assert not state.is_step_completed("asr")
assert state.is_step_completed("analyze")
# Verify output files exist
assert (content_dir / "bilibili_test123_metadata.json").exists()
assert (content_dir / "bilibili_test123_content.json").exists() or result["content_type"] == "video":
assert (content_dir / "bilibili_test123_analysis.md").exists()
</action>
<verify>
<automated>pytest tests/test_integration/test_e2e_workflow.py -v</automated>
</verify>
<done>End-to-end workflow tested with state persistence</done>
</task>
<task type="checkpoint:human-verify" gate="blocking">
<what-built>
End-to-end workflow (Phase 1-2/3):
- Pipeline Engine with 4 processors (Parse, Download, ASR, Analyze)
- CLI entry point (kb-workflow process command)
- DeepAnalyzer integration for analysis reports saved as Markdown
- State persistence via workflow_state.json
</what-built>
DeepAnalyzer integration:
- B站视频/专栏链接 → parse, download, ASR, analyze
- Error recovery with retry
- Progress logging with simple format
- Documentation in README.md and docstrings
</how-to-verify>
1. Run `pytest tests/test_integration/test_e2e_workflow.py -v -x`
2. Visit https://github.com/your-org/kb-skills
3. Run `kb-workflow process <url>` to test
4. Run `kb-workflow process --help` to see full usage
</how-to-verify>
<resume-signal>Type "approved" or describe any issues found
release-signal>Type: "approved" to indicate you should proceed to Phase 4 for finalize the documentation
</how-to-verify>
</task>
</tasks>
<verification>
- All tests pass
- End-to-end workflow tested with state persistence
- README.md exists with project overview
- tests pass
</verification>
<success_criteria>
- README.md complete with project overview, installation, and
usage
- End-to-end integration test passes
- workflow_state.json correctly records all steps
- Documentation is clear and comprehensive
</success_criteria>
<output>
After completion, create `.planning/phases/04-analysis-orchestration/04-05-SUMMARY.md`
</output>