feat: 添加 MediaCrawler 整合支持

- kb-zhihu-metadata: 新增 zhihu_article_crawler.py 支持爬取知乎专栏文章
- kb-zhihu-content: 添加 article-from-media 类型支持从 MediaCrawler 数据读取
- 更新 SKILL.md 文档说明新功能
This commit is contained in:
2026-02-27 17:42:21 +08:00
parent 85797d0a89
commit 193d2b375d
4 changed files with 423 additions and 23 deletions
+46 -18
View File
@@ -6,6 +6,7 @@
- Video download using **yt-dlp**
- Subtitle extraction (returns no subtitle, triggers Whisper)
- **Read article content from MediaCrawler data** (NEW!)
- Supports `ZHIHU_COOKIE` environment variable
## Environment Variables
@@ -22,20 +23,60 @@ python zhihu_content.py --id "12345678" --type video --output "/path/to/download
# Extract subtitle (returns no subtitle, triggers Whisper)
python zhihu_content.py --id "12345678" --type subtitle --output "/path/to/subtitles/"
# 从 MediaCrawler 读取文章内容 (⚡ NEW!)
python zhihu_content.py --type article-from-media --input "~/work/MediaCrawler/data/zhihu/json/detail_contents_2026-02-27.json" --output "~/work/knowledge-base/data/zhihu/"
```
## Output
## MediaCrawler 数据读取
**新功能**:从 MediaCrawler 爬取的数据中提取知乎文章内容
### 使用方法
```bash
# 从 MediaCrawler JSON 文件读取文章内容
python zhihu_content.py --type article-from-media \
--input "~/work/MediaCrawler/data/zhihu/json/detail_contents_2026-02-27.json" \
--output "~/work/knowledge-base/data/zhihu/"
# 查找最新的爬取数据
python zhihu_content.py --type article-from-media \
--output "~/work/knowledge-base/data/zhihu/"
```
### 参数说明
| 参数 | 说明 |
|------|------|
| `--type article-from-media` | 从 MediaCrawler 数据读取 |
| `--input` | MediaCrawler 输出文件路径(可选,不指定则自动查找最新) |
| `--output` | 输出目录 |
### 输出格式
```json
{
"success": true,
"type": "video",
"filename": "video_title.mp4",
"filepath": "/path/to/downloads/video_title.mp4",
"message": "下载完成: video_title.mp4"
"type": "article",
"filename": "zhihu_article_如何高效的爬取微信公众号.md",
"filepath": "/home/xxx/work/knowledge-base/data/zhihu/zhihu_article_如何高效的爬取微信公众号.md",
"message": "文章内容已导出"
}
```
## 原有功能
### Video Download
Video download uses **yt-dlp**:
1. Parses `ZHIHU_COOKIE` environment variable (optional)
2. Creates temporary Netscape-format cookie file
3. Runs `yt-dlp -f best -o <output> <url>`
4. Cleans up temporary file
**Note**: 知乎视频通常没有官方字幕,将自动触发 Whisper 语音识别
## Installation
```bash
@@ -43,16 +84,3 @@ python zhihu_content.py --id "12345678" --type subtitle --output "/path/to/subti
# Ensure yt-dlp is installed
pip install yt-dlp aiohttp
```
## How It Works
Video download uses **yt-dlp**:
1. Parses `ZHIHU_COOKIE` environment variable (optional)
2. Creates temporary Netscape-format cookie file
3. Runs `yt-dlp -f best -o <output> <url>`
4. Cleans up temporary file
## Notes
- 知乎视频通常没有官方字幕,将自动触发 Whisper 语音识别
- 视频下载需要能访问知乎(可能需要代理)