---
phase: 08-crawler
plan: 01
subsystem: crawler
tags: [playwright, pydantic, zhihu, api-interception]

requires:
  - phase: 07-login-auth
    provides: ZhihuLogin + Cookie management
provides:
  - ZhihuCrawler class with crawl_answer() and crawl_article() methods
  - AnswerData, ArticleData data models
  - CrawlResult wrapper
affects: [08-02, 08-03]

tech-stack:
  added: []
  patterns:
    - Playwright route interception for content extraction
    - pydantic data models for structured data

key-files:
  created:
    - kb_workflow/crawler/__init__.py
    - kb_workflow/crawler/models.py
    - kb_workflow/crawler/zhihu_crawler.py
    - tests/test_crawler/__init__.py
    - tests/test_crawler/conftest.py
    - tests/test_crawler/test_answer_crawler.py
    - tests/test_crawler/test_article_crawler.py
    - tests/test_crawler/test_metadata_extraction.py
  modified: []
key-decisions:
  - use Playwright page.route() for API interception over direct API calls
  - pydantic models for structured data
  - TDD workflow with mocking for easy test maintenance
requirements-completed: [CONT-01, CONT-02, CONT-04]
duration: 3min
completed: 2026-03-17
---

## Summary

实现了知乎问答和专栏内容爬取功能，使用 Playwright 拦截 API 请求获取结构化 JSON 数据。

### What Was Built

**ZhihuCrawler 类** (`kb_workflow/crawler/zhihu_crawler.py`)
- `crawl_answer(url)` - 爬取知乎问答，返回 `AnswerData`
- `crawl_article(url)` - 爬取知乎专栏，返回 `ArticleData`
- API 拦截器 `ZhihuApiInterceptor` 捕获 `/api/v4/answers/` 和 `/api/v4/articles/` 响应
- DOM 解析备选方案（使用 `ZhihuPlaywrightExtractor`）

**数据模型** (`kb_workflow/crawler/models.py`)
- `AuthorInfo` - 作者信息（名称、URL token、签名）
- `ContentMetadata` - 内容元数据（作者、点赞数、评论数、时间戳）
- `AnswerData` - 问答数据（ID、问题ID、标题、内容、摘要、元数据）
- `ArticleData` - 专栏数据（ID、标题、内容、摘要、元数据）
- `CrawlResult` - 统一结果包装（成功/失败、内容类型、数据/错误）

### Test Coverage

**15 个测试全部通过**
- `test_answer_crawler.py` - 5 个测试
- `test_article_crawler.py` - 4 个测试
- `test_metadata_extraction.py` - 6 个测试

### Commits

1. `a102a75` feat(08-01): create crawler data models and test scaffolding
2. `912773f` test(08-01): add failing tests for ZhihuCrawler
3. `aa96bc9` feat(08-01): implement ZhihuCrawler with API interception

### Key Decisions

1. **API 拦截 vs 直接 API 调用**: 使用 Playwright `page.route()` 拦截浏览器请求，自动处理签名和 Cookie
2. **TDD 工作流**: 先写测试，使用 mock fixtures 确保测试可维护性
3. **备选方案**: 当 API 拦截失败时，回退到 DOM 解析

### Requirements Completed

- **CONT-01**: 爬取知乎问答，获取问题标题、描述、回答内容 ✓
- **CONT-02**: 爬取知乎专栏/文章，获取标题和正文 ✓
- **CONT-04**: 爬取结果包含作者信息、发布时间、点赞数等元数据 ✓

## Self-Check: PASSED