kb-skills/.planning/phases/04-analysis-orchestration/04-01-PLAN.md

---

phase: 04-analysis-orchestration plan: 01 type: execute wave: 1 depends_on: [] files_modified:

• kb_pipeline/processors/base.py
• kb_pipeline/processors/parse.py
• kb_pipeline/processors/__init__.py
• kb_pipeline/schemas.py
• tests/test_processors/test_base.py
• tests/test_processors/test_parse.py autonomous: true requirements: [MNT-01] user_setup: []

must_haves: truths: - "BaseProcessor defines unified interface for all processors" - "ParseProcessor correctly parses Bilibili and Zhihu URLs" - "All processors inherit from BaseProcessor with consistent process() method" artifacts: - path: "kb_pipeline/processors/base.py" provides: "BaseProcessor abstract class" contains: "class BaseProcessor" min_lines: 80 - path: "kb_pipeline/processors/parse.py" provides: "ParseProcessor implementation" contains: "class ParseProcessor" min_lines: 60 - path: "kb_pipeline/schemas.py" provides: "ParseInput/ParseOutput schemas" contains: "class ParseInput" key_links: - from: "kb_pipeline/processors/parse.py" to: "kb_pipeline/link_parser.py" via: "LinkParser.parse()" pattern: "LinkParser().parse" - from: "kb_pipeline/processors/base.py" to: "kb_pipeline/state.py" via: "WorkflowStateManager integration" pattern: "state_manager."

Create unified BaseProcessor interface and ParseProcessor to establish the foundation for the pipeline architecture.

Purpose: Provide consistent processor interface that all steps (parse, download, asr, analyze) will follow, enabling unified state management and error handling. Output: BaseProcessor abstract class, ParseProcessor implementation, and related schemas.

<execution_context> @/home/starmen1265/.claude/get-shit-done/workflows/execute-plan.md @/home/starmen1265/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/04-analysis-orchestration/04-CONTEXT.md @.planning/phases/04-analysis-orchestration/04-RESEARCH.md

From kb_pipeline/schemas.py:

class StepStatus(str, Enum):
    PENDING = "pending"
    RUNNING = "running"
    COMPLETED = "completed"
    FAILED = "failed"
    SKIPPED = "skipped"

class StepInput(BaseModel):
    content_id: str = Field(..., min_length=1)
    platform: Literal["bilibili", "zhihu"]

class StepOutput(BaseModel):
    success: bool
    error_message: Optional[str] = None
    completed_at: datetime = Field(default_factory=datetime.now)

class WorkflowState(BaseModel):
    def is_step_completed(self, step_name: str) -> bool: ...
    def mark_step_started(self, step_name: str) -> None: ...
    def mark_step_completed(self, step_name: str, output_summary: Optional[str] = None) -> None: ...
    def mark_step_failed(self, step_name: str, error_message: str) -> None: ...

From kb_pipeline/state.py:

class WorkflowStateManager:
    STATE_FILENAME = "workflow_state.json"
    def __init__(self, content_dir: Path): ...
    def load(self) -> Optional[WorkflowState]: ...
    def save(self, state: WorkflowState) -> None: ...
    def get_or_create(self, content_id: str, platform: str, source_url: str) -> WorkflowState: ...

From kb_pipeline/link_parser.py:

@dataclass
class ParsedLink:
    platform: str
    content_type: str
    content_id: str
    original_url: str
    resolved_url: str = ""

class LinkParser:
    def parse(self, url: str) -> Optional[ParsedLink]: ...

From kb_pipeline/processors/asr.py (existing pattern reference):

class ASRProcessor:
    STEP_NAME = "asr"
    def __init__(self, state_manager: Optional[WorkflowStateManager] = None):
        self.settings = get_settings()
        self.state_manager = state_manager
    async def process(self, input_data: ASRInput) -> ASROutput: ...

Task 1: Define BaseProcessor abstract class kb_pipeline/processors/base.py, tests/test_processors/test_base.py - Test 1: BaseProcessor cannot be instantiated directly (abstract) - Test 2: Subclass must implement process() method - Test 3: BaseProcessor provides state_manager attribute - Test 4: BaseProcessor provides STEP_NAME class attribute Create `kb_pipeline/processors/base.py` with BaseProcessor abstract class:

1. Define abstract base class with:

   • STEP_NAME: str class attribute (must be overridden by subclass)
   • init(self, state_manager: Optional[WorkflowStateManager] = None)
   • settings: KBSettings from get_settings()
   • @abstractmethod process(self, input_data: StepInput) -> StepOutput

2. Use ABC (Abstract Base Class) from abc module

3. Import types from typing: Optional, TypeVar, Generic

4. Keep it minimal - no complex template methods, just the interface contract

Reference existing ASRProcessor pattern but extract the common interface. Do NOT add retry logic - that belongs in concrete implementations. Do NOT add complex inheritance hierarchies - keep it flat and simple. pytest tests/test_processors/test_base.py -v -x BaseProcessor abstract class exists with process() method, cannot be instantiated directly

Task 2: Add ParseInput and ParseOutput schemas kb_pipeline/schemas.py, tests/test_pipeline/test_schemas.py - Test 1: ParseInput validates url field (required, min_length=1) - Test 2: ParseOutput contains platform, content_type, content_id, resolved_url - Test 3: ParseOutput inherits success and error_message from StepOutput Add ParseInput and ParseOutput schemas to `kb_pipeline/schemas.py`:

class ParseInput(StepInput):
    """Parse step input - URL to parse"""
    url: str = Field(..., min_length=1, description="URL to parse")
    # Override content_id and platform as optional for parse step
    content_id: str = Field(default="", description="Will be populated after parsing")
    platform: str = Field(default="", description="Will be populated after parsing")

class ParseOutput(StepOutput):
    """Parse step output"""
    platform: str = Field(..., description="Detected platform")
    content_type: str = Field(..., description="Content type (video, article, etc.)")
    content_id: str = Field(..., description="Extracted content ID")
    resolved_url: str = Field(..., description="Resolved URL after short link expansion")
    original_url: str = Field(..., description="Original URL provided by user")

Place after ASROutput schema (around line 236). Add tests to existing test_schemas.py file. pytest tests/test_pipeline/test_schemas.py -v -x -k "Parse" ParseInput and ParseOutput schemas exist and validate correctly

Task 3: Implement ParseProcessor with __init__.py export kb_pipeline/processors/parse.py, kb_pipeline/processors/__init__.py, tests/test_processors/test_parse.py - Test 1: ParseProcessor parses Bilibili video URL correctly - Test 2: ParseProcessor parses Zhihu answer URL correctly - Test 3: ParseProcessor returns ParseOutput with success=False for unknown URLs - Test 4: ParseProcessor records state when state_manager is provided - Test 5: ParseProcessor can be imported from kb_pipeline.processors Create `kb_pipeline/processors/parse.py`:

1. Import BaseProcessor from .base, LinkParser from kb_pipeline.link_parser

2. Import ParseInput, ParseOutput from kb_pipeline.schemas

3. Implement ParseProcessor(BaseProcessor):

   • STEP_NAME = "parse"
   • async def process(self, input_data: ParseInput) -> ParseOutput: a. Create LinkParser instance b. Call parser.parse(input_data.url) c. If parsed is None, return ParseOutput(success=False, error_message="Unsupported URL") d. If parsed, return ParseOutput with parsed fields e. Record state on success (if state_manager exists)

4. Keep implementation simple - just wrap LinkParser

5. Handle both success and failure cases with proper state recording

Then update kb_pipeline/processors/__init__.py to export new classes:

from .base import BaseProcessor
from .parse import ParseProcessor
from .asr import ASRProcessor

__all__ = ["BaseProcessor", "ParseProcessor", "ASRProcessor"]

Keep existing ASRProcessor export and add new ones. pytest tests/test_processors/test_parse.py -v -x && python -c "from kb_pipeline.processors import BaseProcessor, ParseProcessor, ASRProcessor; print('OK')" ParseProcessor correctly parses URLs, returns structured ParseOutput, and is exported from processors module

- All tests pass: `pytest tests/test_processors/test_base.py tests/test_processors/test_parse.py -v` - BaseProcessor is abstract and cannot be instantiated - ParseProcessor correctly wraps LinkParser - State recording works when state_manager is provided - All processors can be imported from kb_pipeline.processors

<success_criteria>

• BaseProcessor abstract class defines unified interface
• ParseProcessor inherits from BaseProcessor
• ParseInput/ParseOutput schemas exist and validate
• All tests pass
• Code follows existing patterns from ASRProcessor </success_criteria>

After completion, create `.planning/phases/04-analysis-orchestration/04-01-SUMMARY.md`