feat(kb-deep-analyzer): Support GLM-4 with custom base URL
- Support AI_BASE_URL environment variable for custom API endpoint - Add --base-url CLI argument - Fix zhipu API endpoint to use coding/paas/v4 path
This commit is contained in:
+100
-14
@@ -1,23 +1,109 @@
|
|||||||
# 2026-02-10
|
# 2026-02-10
|
||||||
|
|
||||||
## 知识库技能重构 - B站专栏内容获取
|
## 知识库技能重构 - 完整功能
|
||||||
|
|
||||||
### kb-bilibili-content 专栏内容获取
|
### kb-bilibili-content 专栏内容获取 ✅
|
||||||
|
|
||||||
**测试结果:**
|
|
||||||
```
|
|
||||||
标题: [UEPCG][新手向]复刻ChrisMurphy的城市道路建筑生成...
|
|
||||||
内容长度: 12659 字符
|
|
||||||
✅ 获取成功
|
|
||||||
```
|
|
||||||
|
|
||||||
**实现方式:**
|
|
||||||
- 使用 Playwright 异步API爬取专栏页面
|
- 使用 Playwright 异步API爬取专栏页面
|
||||||
- 选择器: `.read-article-holder`
|
- 选择器: `.read-article-holder`
|
||||||
- 获取完整文章正文
|
- 获取完整文章正文
|
||||||
|
|
||||||
### 已提交
|
### Git版本管理和清理 ✅ (2026-02-10下午)
|
||||||
|
|
||||||
- `47355b3` - feat(kb-bilibili-content): Add article content extraction using Playwright
|
**需求1: Git版本管理**
|
||||||
- `56907e3` - refactor(kb-zhihu-metadata): Simplify to only support zvideo and zhuanlan
|
- Git仓库: `archive/` 目录
|
||||||
- 使用playurl API,无需Cookie即可下载视频
|
- 配置环境变量:
|
||||||
|
- `KB_GIT_ROOT` - Git仓库目录
|
||||||
|
- `KB_GIT_REMOTE` - 远程仓库地址
|
||||||
|
- `KB_GIT_USER` - 用户名
|
||||||
|
- `KB_GIT_EMAIL` - 邮箱
|
||||||
|
- 命令: `--trigger git-init`, `--trigger git-status`
|
||||||
|
- 推送重试: 3次
|
||||||
|
|
||||||
|
**需求2: 订阅数据管理**
|
||||||
|
- 存储位置: `archive/subscriptions/`
|
||||||
|
- 文件: `subscriptions.json`, `processed.json`
|
||||||
|
- Git跟踪历史,防止误删
|
||||||
|
|
||||||
|
**需求3: 自动清理策略**
|
||||||
|
| 类型 | 保留 | 清理周期 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 分析报告 | ✅ | 永久 |
|
||||||
|
| 链接信息 | ✅ | 永久 |
|
||||||
|
| 元数据 | ✅ | 永久 |
|
||||||
|
| 订阅数据 | ✅ | Git管理 |
|
||||||
|
| 视频文件 | ❌ | 7天 |
|
||||||
|
| 截帧文件 | ❌ | 7天 |
|
||||||
|
|
||||||
|
**命令:**
|
||||||
|
```bash
|
||||||
|
# 查看状态
|
||||||
|
python kb_workflow.py --trigger status
|
||||||
|
|
||||||
|
# 处理待处理内容
|
||||||
|
python kb_workflow.py --trigger manual --scope pending --limit 5
|
||||||
|
|
||||||
|
# 初始化Git
|
||||||
|
python kb_workflow.py --trigger git-init
|
||||||
|
|
||||||
|
# 清理过期文件
|
||||||
|
python kb_workflow.py --trigger cleanup --days 30
|
||||||
|
```
|
||||||
|
|
||||||
|
### 深度分析模板增强 ✅
|
||||||
|
|
||||||
|
- 完整元数据保留 (bvid, cvid, mid, platform)
|
||||||
|
- AI生成标签 (5个)
|
||||||
|
- 模板支持环境变量覆盖
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 待处理事项
|
||||||
|
|
||||||
|
### 重要
|
||||||
|
- [ ] 配置 OpenRouter API
|
||||||
|
- [ ] 配置阿里云 TTS
|
||||||
|
- [ ] 🔒 安全巡检报告
|
||||||
|
|
||||||
|
### 字幕增强(待实现)
|
||||||
|
- [ ] 视觉大模型方案 (GPT-4V/Claude/GLM-4V)
|
||||||
|
- [ ] 语音识别方案 (Whisper/ASR)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 已提交Commit
|
||||||
|
|
||||||
|
| Commit | 说明 |
|
||||||
|
|--------|------|
|
||||||
|
| `38f1624` | docs: Update TODO |
|
||||||
|
| `f41fb61` | feat(kb-workflow): Add Git version control and cleanup |
|
||||||
|
| `8b43416` | feat(kb-deep-analyzer): Enhance template and metadata |
|
||||||
|
| `f2309bf` | feat(kb-workflow): Complete file storage |
|
||||||
|
| `3f74bcd` | docs: Add subtitle enhancement tasks |
|
||||||
|
| `47355b3` | feat(kb-bilibili-content): Article extraction |
|
||||||
|
| `56907e3` | refactor(kb-zhihu-metadata): Simplify |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 测试要点
|
||||||
|
|
||||||
|
1. **kb-link-parser** - 链接解析 ✅
|
||||||
|
2. **kb-bilibili-metadata** - B站元数据 ✅
|
||||||
|
3. **kb-zhihu-metadata** - 知乎元数据 (zvideo/zhuanlan) ⚠️
|
||||||
|
4. **kb-bilibili-content** - 内容获取 ✅
|
||||||
|
5. **kb-ocr-processor** - OCR识别 ✅
|
||||||
|
6. **kb-deep-analyzer** - 深度分析 ✅ (mock模式)
|
||||||
|
7. **kb-workflow** - 工作流编排 ✅
|
||||||
|
|
||||||
|
### Git配置
|
||||||
|
```bash
|
||||||
|
export KB_GIT_ROOT="/path/to/archive"
|
||||||
|
export KB_GIT_REMOTE="git@github.com:user/repo.git"
|
||||||
|
export KB_GIT_USER="Username"
|
||||||
|
export KB_GIT_EMAIL="user@example.com"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 清理命令
|
||||||
|
```bash
|
||||||
|
python kb_workflow.py --trigger cleanup --days 30
|
||||||
|
```
|
||||||
|
|||||||
@@ -109,13 +109,14 @@ class SimpleAIClient:
|
|||||||
def _call_zhipu(self, prompt: str, max_tokens: int) -> str:
|
def _call_zhipu(self, prompt: str, max_tokens: int) -> str:
|
||||||
"""调用智谱 AI API"""
|
"""调用智谱 AI API"""
|
||||||
import requests
|
import requests
|
||||||
base_url = self.base_url or "https://open.bigmodel.cn/api/paas/v4"
|
base_url = self.base_url or "https://open.bigmodel.cn/api/coding/paas/v4"
|
||||||
url = f"{base_url}/chat/completions"
|
url = f"{base_url}/chat/completions"
|
||||||
|
|
||||||
headers = {"Authorization": f"Bearer {self.api_key}"}
|
headers = {"Authorization": f"Bearer {self.api_key}"}
|
||||||
payload = {
|
payload = {
|
||||||
"model": self.model,
|
"model": self.model,
|
||||||
"messages": [{"role": "user", "content": prompt}]
|
"messages": [{"role": "user", "content": prompt}],
|
||||||
|
"max_tokens": max_tokens
|
||||||
}
|
}
|
||||||
|
|
||||||
response = requests.post(url, headers=headers, json=payload, timeout=30)
|
response = requests.post(url, headers=headers, json=payload, timeout=30)
|
||||||
@@ -743,6 +744,8 @@ def main():
|
|||||||
help="AI 提供商 (默认: mock)")
|
help="AI 提供商 (默认: mock)")
|
||||||
parser.add_argument("--model", default="gpt-4",
|
parser.add_argument("--model", default="gpt-4",
|
||||||
help="AI 模型 (默认: gpt-4)")
|
help="AI 模型 (默认: gpt-4)")
|
||||||
|
parser.add_argument("--base-url", "-u", default=None,
|
||||||
|
help="API Base URL (或设置 AI_BASE_URL 环境变量)")
|
||||||
parser.add_argument("--no-frontmatter", action="store_true",
|
parser.add_argument("--no-frontmatter", action="store_true",
|
||||||
help="不生成 frontmatter")
|
help="不生成 frontmatter")
|
||||||
|
|
||||||
@@ -757,9 +760,10 @@ def main():
|
|||||||
|
|
||||||
# 初始化
|
# 初始化
|
||||||
ai_client = SimpleAIClient(
|
ai_client = SimpleAIClient(
|
||||||
api_key=args.api_key,
|
api_key=args.api_key or os.environ.get("AI_API_KEY", ""),
|
||||||
provider=args.provider,
|
provider=args.provider or os.environ.get("AI_PROVIDER", "mock"),
|
||||||
model=args.model
|
model=args.model or os.environ.get("AI_MODEL", "gpt-4"),
|
||||||
|
base_url=args.base_url or os.environ.get("AI_BASE_URL", "")
|
||||||
)
|
)
|
||||||
analyzer = DeepAnalyzer(ai_client)
|
analyzer = DeepAnalyzer(ai_client)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user