自动同步: 2026-03-14 02:10
This commit is contained in:
@@ -1,101 +1,207 @@
|
||||
---
|
||||
name: kb-zhihu-metadata
|
||||
description: 知乎元数据提取 - 获取问题/回答/专栏/视频的详细信息
|
||||
---
|
||||
|
||||
# KB Zhihu Metadata Skill
|
||||
|
||||
**kb-zhihu-metadata** - Extract Zhihu question/answer/user metadata and crawl articles.
|
||||
**kb-zhihu-metadata** - 提取知乎问题、回答、用户、专栏的元数据。
|
||||
|
||||
## Features
|
||||
## 什么时候使用
|
||||
|
||||
- Question metadata extraction
|
||||
- Answer metadata extraction
|
||||
- User information extraction
|
||||
- **Article crawling using MediaCrawler** (NEW!)
|
||||
- JSON output format
|
||||
- Markdown conversion support
|
||||
- **Cookie 支持**:可配置 ZHIHU_COOKIE 访问需要登录的内容
|
||||
- kb-workflow 步骤 2:获取知乎内容的元数据
|
||||
- 需要问题标题、回答者、文章标题等信息
|
||||
|
||||
## 功能
|
||||
|
||||
- 问题元数据提取
|
||||
- 回答元数据提取
|
||||
- 用户信息提取
|
||||
- 专栏文章爬取(使用 MediaCrawler)
|
||||
- 支持 `ZHIHU_COOKIE` 环境变量
|
||||
|
||||
## 环境变量
|
||||
|
||||
| 变量名 | 用途 | 必需 |
|
||||
| 变量名 | 必需 | 说明 |
|
||||
|--------|------|------|
|
||||
| `ZHIHU_COOKIE` | 知乎登录 Cookie | 推荐(访问专栏等需要) |
|
||||
| `ZHIHU_COOKIE` | 推荐 | 知乎登录 Cookie(访问专栏等需要) |
|
||||
|
||||
**获取 ZHIHU_COOKIE**:
|
||||
**获取 Cookie:**
|
||||
1. 登录知乎网页版
|
||||
2. 打开开发者工具 (F12) → Application → Cookies → zhihu.com
|
||||
2. F12 → Application → Cookies → zhihu.com
|
||||
3. 复制 Cookie 字符串
|
||||
|
||||
```bash
|
||||
# 添加到 ~/.kb_env
|
||||
echo 'export ZHIHU_COOKIE="你的知乎Cookie"' >> ~/.kb_env
|
||||
source ~/.kb_env
|
||||
export ZHIHU_COOKIE="你的知乎Cookie"
|
||||
```
|
||||
|
||||
## Commands
|
||||
## 命令
|
||||
|
||||
```bash
|
||||
# Question metadata
|
||||
python zhihu_metadata.py --id "266309957" --type question
|
||||
# 问题元数据
|
||||
python3 zhihu_metadata.py --id "266309957" --type question
|
||||
|
||||
# Answer metadata
|
||||
python zhihu_metadata.py --id "2003427891925509507" --type answer
|
||||
# 回答元数据
|
||||
python3 zhihu_metadata.py --id "2003427891925509507" --type answer
|
||||
|
||||
# 专栏文章爬取 (使用 MediaCrawler) ⚡ NEW!
|
||||
python zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/1894054073625461813" --output "~/work/knowledge-base/data/zhihu/"
|
||||
# 知乎视频
|
||||
python3 zhihu_metadata.py --id "123456789" --type zvideo
|
||||
|
||||
# 专栏文章爬取(使用 MediaCrawler)
|
||||
python3 zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/1894054073625461813" --output "~/work/knowledge-base/data/zhihu/"
|
||||
```
|
||||
|
||||
## MediaCrawler 爬取文章
|
||||
## 输出格式 Schema
|
||||
|
||||
**新功能**:使用 MediaCrawler 爬取知乎专栏文章
|
||||
### 必需字段
|
||||
|
||||
### 环境要求
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `success` | boolean | 是否提取成功 |
|
||||
| `platform` | string | 固定为 `zhihu` |
|
||||
| `content_id` | string | 内容 ID |
|
||||
| `content_type` | string | 内容类型 |
|
||||
| `title` | string | 标题 |
|
||||
|
||||
MediaCrawler 已安装在 `~/work/MediaCrawler`
|
||||
### 可选字段
|
||||
|
||||
### 使用方法
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| `author` | string | 作者名称 |
|
||||
| `excerpt` | string | 摘要 |
|
||||
| `stats` | object | 统计数据 |
|
||||
| `output_file` | string | 输出文件路径 |
|
||||
| `error` | string | 错误信息 |
|
||||
|
||||
```bash
|
||||
# 爬取单篇文章
|
||||
python zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/XXXXX"
|
||||
## 成功示例
|
||||
|
||||
# 指定输出目录
|
||||
python zhihu_article_crawler.py --url "https://zhuanlan.zhihu.com/p/XXXXX" --output "/path/to/output/"
|
||||
|
||||
# 输出示例
|
||||
{
|
||||
"success": true,
|
||||
"type": "article",
|
||||
"title": "如何高效的爬取微信公众号",
|
||||
"filepath": "/home/xxx/work/knowledge-base/data/zhihu/zhihu_article_如何高效的爬取微信公众号.md",
|
||||
"content_length": 4563,
|
||||
"message": "爬取成功: 如何高效的爬取微信公众号"
|
||||
}
|
||||
```
|
||||
|
||||
### 工作原理
|
||||
|
||||
1. 调用 MediaCrawler 爬取文章
|
||||
2. 从 JSON 输出中提取内容
|
||||
3. 转换为 Markdown 格式(带 frontmatter)
|
||||
4. 保存到指定目录
|
||||
|
||||
## Output
|
||||
### 问题元数据
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"data": {
|
||||
"platform": "zhihu",
|
||||
"type": "question",
|
||||
"id": "266309957",
|
||||
"title": "问题标题",
|
||||
"author": "作者名字",
|
||||
"platform": "zhihu",
|
||||
"content_type": "question",
|
||||
"content_id": "266309957",
|
||||
"title": "问题标题",
|
||||
"excerpt": "问题描述...",
|
||||
"stats": {
|
||||
"answer_count": 50,
|
||||
"followers": 1000
|
||||
}
|
||||
"follower_count": 1000,
|
||||
"view_count": 50000
|
||||
},
|
||||
"output_file": "metadata/zhihu_266309957.meta.json"
|
||||
}
|
||||
```
|
||||
|
||||
## Installation
|
||||
### 回答元数据
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"platform": "zhihu",
|
||||
"content_type": "answer",
|
||||
"content_id": "2003427891925509507",
|
||||
"title": "问题标题(回答所属问题)",
|
||||
"author": "回答者名字",
|
||||
"excerpt": "回答摘要...",
|
||||
"stats": {
|
||||
"voteup_count": 1000,
|
||||
"comment_count": 50
|
||||
},
|
||||
"output_file": "metadata/zhihu_2003427891925509507.meta.json"
|
||||
}
|
||||
```
|
||||
|
||||
### 专栏文章
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"platform": "zhihu",
|
||||
"content_type": "zhuanlan",
|
||||
"content_id": "1894054073625461813",
|
||||
"title": "文章标题",
|
||||
"author": "作者名字",
|
||||
"excerpt": "文章摘要...",
|
||||
"output_file": "metadata/zhihu_1894054073625461813.meta.json"
|
||||
}
|
||||
```
|
||||
|
||||
### 知乎视频
|
||||
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"platform": "zhihu",
|
||||
"content_type": "zvideo",
|
||||
"content_id": "123456789",
|
||||
"title": "视频标题",
|
||||
"author": "作者名字",
|
||||
"duration": "10:30",
|
||||
"output_file": "metadata/zhihu_123456789.meta.json"
|
||||
}
|
||||
```
|
||||
|
||||
## 错误示例
|
||||
|
||||
### 内容不存在
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"platform": "zhihu",
|
||||
"content_id": "invalid_id",
|
||||
"content_type": "question",
|
||||
"error": "问题不存在或已被删除"
|
||||
}
|
||||
```
|
||||
|
||||
### 需要登录
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"platform": "zhihu",
|
||||
"content_id": "1894054073625461813",
|
||||
"content_type": "zhuanlan",
|
||||
"error": "需要登录才能访问",
|
||||
"suggestion": "请配置 ZHIHU_COOKIE 环境变量"
|
||||
}
|
||||
```
|
||||
|
||||
### 网络错误
|
||||
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"error": "网络请求失败: Connection timeout"
|
||||
}
|
||||
```
|
||||
|
||||
## 安装
|
||||
|
||||
```bash
|
||||
cp -r kb-zhihu-metadata/ ~/skills/
|
||||
cp -r kb-zhihu-metadata/ ~/.openclaw/skills/
|
||||
pip install httpx
|
||||
```
|
||||
|
||||
## 在 kb-workflow 中的使用
|
||||
|
||||
```bash
|
||||
# 步骤 2:元数据提取(知乎问题)
|
||||
python3 ~/.openclaw/skills/kb-zhihu-metadata/zhihu_metadata.py \
|
||||
--id "266309957" --type question \
|
||||
--output "metadata/zhihu_266309957.meta.json"
|
||||
|
||||
# 步骤 2:元数据提取(知乎视频)
|
||||
python3 ~/.openclaw/skills/kb-zhihu-metadata/zhihu_metadata.py \
|
||||
--id "123456789" --type zvideo \
|
||||
--output "metadata/zhihu_123456789.meta.json"
|
||||
```
|
||||
|
||||
## 注意事项
|
||||
|
||||
- 专栏文章可能需要登录才能访问
|
||||
- MediaCrawler 需要单独安装:`~/work/MediaCrawler`
|
||||
- 部分内容可能有访问限制
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Reference in New Issue
Block a user