feat: 添加 MediaCrawler 整合支持
- kb-zhihu-metadata: 新增 zhihu_article_crawler.py 支持爬取知乎专栏文章 - kb-zhihu-content: 添加 article-from-media 类型支持从 MediaCrawler 数据读取 - 更新 SKILL.md 文档说明新功能
This commit is contained in:
+46
-18
@@ -6,6 +6,7 @@
|
||||
|
||||
- Video download using **yt-dlp**
|
||||
- Subtitle extraction (returns no subtitle, triggers Whisper)
|
||||
- **Read article content from MediaCrawler data** (NEW!)
|
||||
- Supports `ZHIHU_COOKIE` environment variable
|
||||
|
||||
## Environment Variables
|
||||
@@ -22,20 +23,60 @@ python zhihu_content.py --id "12345678" --type video --output "/path/to/download
|
||||
|
||||
# Extract subtitle (returns no subtitle, triggers Whisper)
|
||||
python zhihu_content.py --id "12345678" --type subtitle --output "/path/to/subtitles/"
|
||||
|
||||
# 从 MediaCrawler 读取文章内容 (⚡ NEW!)
|
||||
python zhihu_content.py --type article-from-media --input "~/work/MediaCrawler/data/zhihu/json/detail_contents_2026-02-27.json" --output "~/work/knowledge-base/data/zhihu/"
|
||||
```
|
||||
|
||||
## Output
|
||||
## MediaCrawler 数据读取
|
||||
|
||||
**新功能**:从 MediaCrawler 爬取的数据中提取知乎文章内容
|
||||
|
||||
### 使用方法
|
||||
|
||||
```bash
|
||||
# 从 MediaCrawler JSON 文件读取文章内容
|
||||
python zhihu_content.py --type article-from-media \
|
||||
--input "~/work/MediaCrawler/data/zhihu/json/detail_contents_2026-02-27.json" \
|
||||
--output "~/work/knowledge-base/data/zhihu/"
|
||||
|
||||
# 查找最新的爬取数据
|
||||
python zhihu_content.py --type article-from-media \
|
||||
--output "~/work/knowledge-base/data/zhihu/"
|
||||
```
|
||||
|
||||
### 参数说明
|
||||
|
||||
| 参数 | 说明 |
|
||||
|------|------|
|
||||
| `--type article-from-media` | 从 MediaCrawler 数据读取 |
|
||||
| `--input` | MediaCrawler 输出文件路径(可选,不指定则自动查找最新) |
|
||||
| `--output` | 输出目录 |
|
||||
|
||||
### 输出格式
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"type": "video",
|
||||
"filename": "video_title.mp4",
|
||||
"filepath": "/path/to/downloads/video_title.mp4",
|
||||
"message": "下载完成: video_title.mp4"
|
||||
"type": "article",
|
||||
"filename": "zhihu_article_如何高效的爬取微信公众号.md",
|
||||
"filepath": "/home/xxx/work/knowledge-base/data/zhihu/zhihu_article_如何高效的爬取微信公众号.md",
|
||||
"message": "文章内容已导出"
|
||||
}
|
||||
```
|
||||
|
||||
## 原有功能
|
||||
|
||||
### Video Download
|
||||
|
||||
Video download uses **yt-dlp**:
|
||||
1. Parses `ZHIHU_COOKIE` environment variable (optional)
|
||||
2. Creates temporary Netscape-format cookie file
|
||||
3. Runs `yt-dlp -f best -o <output> <url>`
|
||||
4. Cleans up temporary file
|
||||
|
||||
**Note**: 知乎视频通常没有官方字幕,将自动触发 Whisper 语音识别
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
@@ -43,16 +84,3 @@ python zhihu_content.py --id "12345678" --type subtitle --output "/path/to/subti
|
||||
# Ensure yt-dlp is installed
|
||||
pip install yt-dlp aiohttp
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
Video download uses **yt-dlp**:
|
||||
1. Parses `ZHIHU_COOKIE` environment variable (optional)
|
||||
2. Creates temporary Netscape-format cookie file
|
||||
3. Runs `yt-dlp -f best -o <output> <url>`
|
||||
4. Cleans up temporary file
|
||||
|
||||
## Notes
|
||||
|
||||
- 知乎视频通常没有官方字幕,将自动触发 Whisper 语音识别
|
||||
- 视频下载需要能访问知乎(可能需要代理)
|
||||
|
||||
Reference in New Issue
Block a user