docs(04): complete phase planning
This commit is contained in:
@@ -6,6 +6,7 @@ wave: 2
|
||||
depends_on: [04-01]
|
||||
files_modified:
|
||||
- kb_pipeline/processors/download.py
|
||||
- kb_pipeline/processors/__init__.py
|
||||
- kb_pipeline/schemas.py
|
||||
- tests/test_processors/test_download.py
|
||||
autonomous: true
|
||||
@@ -37,7 +38,7 @@ must_haves:
|
||||
- from: "kb_pipeline/processors/download.py"
|
||||
to: "kb_pipeline/state.py"
|
||||
via: "WorkflowStateManager"
|
||||
pattern: "state_manager\\."
|
||||
pattern: "state_manager\."
|
||||
---
|
||||
|
||||
<objective>
|
||||
@@ -138,7 +139,7 @@ Follow StepInput/StepOutput pattern from Phase 1.
|
||||
Create test file with schema validation tests.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>pytest tests/test_processors/test_download.py -v -x</automated>
|
||||
<automated>pytest tests/test_processors/test_download.py -v -x -k "test_schema"</automated>
|
||||
</verify>
|
||||
<done>DownloadInput/DownloadOutput schemas defined and validated with tests</done>
|
||||
</task>
|
||||
@@ -175,21 +176,22 @@ Create `kb_pipeline/processors/download.py` with DownloadProcessor class.
|
||||
Create comprehensive tests mocking adapters and state_manager.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>pytest tests/test_processors/test_download.py -v -x</automated>
|
||||
<automated>pytest tests/test_processors/test_download.py -v -x -k "test_metadata"</automated>
|
||||
</verify>
|
||||
<done>DownloadProcessor fetches metadata with state recording</done>
|
||||
</task>
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>Task 3: Add content downloading to DownloadProcessor</name>
|
||||
<files>kb_pipeline/processors/download.py, tests/test_processors/test_download.py</files>
|
||||
<name>Task 3: Add content downloading and __init__.py export</name>
|
||||
<files>kb_pipeline/processors/download.py, kb_pipeline/processors/__init__.py, tests/test_processors/test_download.py</files>
|
||||
<behavior>
|
||||
- Test 1: DownloadProcessor downloads Bilibili article content
|
||||
- Test 2: DownloadProcessor skips video download (handled by ASR)
|
||||
- Test 3: DownloadProcessor creates correct output file paths
|
||||
- Test 4: DownloadProcessor can be imported from kb_pipeline.processors
|
||||
</behavior>
|
||||
<action>
|
||||
Extend DownloadProcessor to download content.
|
||||
Extend DownloadProcessor to download content and update __init__.py.
|
||||
|
||||
**For articles/text content:**
|
||||
- Call adapter.fetch_content() to get text
|
||||
@@ -205,18 +207,8 @@ Extend DownloadProcessor to download content.
|
||||
- Content: `{platform}_{content_id}_content.md` (for articles)
|
||||
|
||||
Update tests to verify content downloading behavior.
|
||||
</action>
|
||||
<verify>
|
||||
<automated>pytest tests/test_processors/test_download.py -v -x</automated>
|
||||
</verify>
|
||||
<done>DownloadProcessor downloads content for text-based content types</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 4: Update processors __init__.py</name>
|
||||
<files>kb_pipeline/processors/__init__.py</files>
|
||||
<action>
|
||||
Update `kb_pipeline/processors/__init__.py` to export DownloadProcessor.
|
||||
Then update `kb_pipeline/processors/__init__.py` to export DownloadProcessor:
|
||||
|
||||
```python
|
||||
from .base import BaseProcessor
|
||||
@@ -228,9 +220,9 @@ __all__ = ["BaseProcessor", "ParseProcessor", "DownloadProcessor", "ASRProcessor
|
||||
```
|
||||
</action>
|
||||
<verify>
|
||||
<automated>python -c "from kb_pipeline.processors import DownloadProcessor; print('OK')"</automated>
|
||||
<automated>pytest tests/test_processors/test_download.py -v -x && python -c "from kb_pipeline.processors import DownloadProcessor; print('OK')"</automated>
|
||||
</verify>
|
||||
<done>DownloadProcessor exported from processors module</done>
|
||||
<done>DownloadProcessor downloads content for text-based content types and is exported from processors module</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
@@ -240,6 +232,7 @@ __all__ = ["BaseProcessor", "ParseProcessor", "DownloadProcessor", "ASRProcessor
|
||||
- DownloadProcessor correctly uses PlatformAdapter implementations
|
||||
- State recording works for both success and failure cases
|
||||
- Content files are saved to correct locations
|
||||
- DownloadProcessor can be imported from kb_pipeline.processors
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
|
||||
Reference in New Issue
Block a user