starmen1265
193d2b375d
- kb-zhihu-metadata: 新增 zhihu_article_crawler.py 支持爬取知乎专栏文章 - kb-zhihu-content: 添加 article-from-media 类型支持从 MediaCrawler 数据读取 - 更新 SKILL.md 文档说明新功能
102 lines
2.3 KiB
Markdown
102 lines
2.3 KiB
Markdown
# KB Zhihu Metadata Skill
|
||
|
||
**kb-zhihu-metadata** - Extract Zhihu question/answer/user metadata and crawl articles.
|
||
|
||
## Features
|
||
|
||
- Question metadata extraction
|
||
- Answer metadata extraction
|
||
- User information extraction
|
||
- **Article crawling using MediaCrawler** (NEW!)
|
||
- JSON output format
|
||
- Markdown conversion support
|
||
- **Cookie 支持**：可配置 ZHIHU_COOKIE 访问需要登录的内容
|
||
|
||
## 环境变量
|
||
|
||
| 变量名 | 用途 | 必需 |
|
||
|--------|------|------|
|
||
| `ZHIHU_COOKIE` | 知乎登录 Cookie | 推荐（访问专栏等需要） |
|
||
|
||
**获取 ZHIHU_COOKIE**：
|
||
1. 登录知乎网页版
|
||
2. 打开开发者工具 (F12) → Application → Cookies → zhihu.com
|
||
3. 复制 Cookie 字符串
|
||
|
||
```bash
|
||
# 添加到 ~/.kb_env
|
||
echo 'export ZHIHU_COOKIE="你的知乎Cookie"' >> ~/.kb_env
|
||
source ~/.kb_env
|
||
```
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
# Question metadata
|
||
python zhihu_metadata.py --id "266309957" --type question
|
||
|
||
# Answer metadata
|
||
python zhihu_metadata.py --id "2003427891925509507" --type answer
|
||
|
||
# 专栏文章爬取 (使用 MediaCrawler) ⚡ NEW!
|
||
python zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/1894054073625461813" --output "~/work/knowledge-base/data/zhihu/"
|
||
```
|
||
|
||
## MediaCrawler 爬取文章
|
||
|
||
**新功能**：使用 MediaCrawler 爬取知乎专栏文章
|
||
|
||
### 环境要求
|
||
|
||
MediaCrawler 已安装在 `~/work/MediaCrawler`
|
||
|
||
### 使用方法
|
||
|
||
```bash
|
||
# 爬取单篇文章
|
||
python zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/XXXXX"
|
||
|
||
# 指定输出目录
|
||
python zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/XXXXX" --output "/path/to/output/"
|
||
|
||
# 输出示例
|
||
{
|
||
"success": true,
|
||
"type": "article",
|
||
"title": "如何高效的爬取微信公众号",
|
||
"filepath": "/home/xxx/work/knowledge-base/data/zhihu/zhihu_article_如何高效的爬取微信公众号.md",
|
||
"content_length": 4563,
|
||
"message": "爬取成功: 如何高效的爬取微信公众号"
|
||
}
|
||
```
|
||
|
||
### 工作原理
|
||
|
||
1. 调用 MediaCrawler 爬取文章
|
||
2. 从 JSON 输出中提取内容
|
||
3. 转换为 Markdown 格式（带 frontmatter）
|
||
4. 保存到指定目录
|
||
|
||
## Output
|
||
|
||
```json
|
||
{
|
||
"success": true,
|
||
"data": {
|
||
"platform": "zhihu",
|
||
"type": "question",
|
||
"id": "266309957",
|
||
"title": "问题标题",
|
||
"author": "作者名字",
|
||
"answer_count": 50,
|
||
"followers": 1000
|
||
}
|
||
}
|
||
```
|
||
|
||
## Installation
|
||
|
||
```bash
|
||
cp -r kb-zhihu-metadata/ ~/skills/
|
||
```
|