08066334c4
feat(kb-deep-analyzer): Smart subtitle chunking strategy
...
## 改进内容
### 动态分段
- 根据字符数动态计算分段数
- 每个chunk最多3000字符
- 限制最小3段，最多20段
### 自然断点分割
- 优先级：段落(双换行) > 句末(。\n) > 句号 > 换行
- 不截断，保持语义完整
### 时间估算
- 用字符比例估算每段时间
- 长视频时间轴更准确
2026-02-10 16:41:15 +08:00
3cfd36cf4e
fix(kb-deep-analyzer): Fix video analysis threshold
...
- Change threshold from LONG_VIDEO_THRESHOLD (30min) to SHORT_VIDEO_THRESHOLD (10min)
- Now videos 10+ minutes use chunked analysis for deeper insights
2026-02-10 16:30:41 +08:00
db9bcc20c9
docs: Add GLM-4 test results and configuration
2026-02-10 16:29:38 +08:00
3faf121edb
feat(kb-deep-analyzer): Support GLM-4 with custom base URL
...
- Support AI_BASE_URL environment variable for custom API endpoint
- Add --base-url CLI argument
- Fix zhipu API endpoint to use coding/paas/v4 path
2026-02-10 16:24:31 +08:00
38f1624b05
docs: Update TODO with completed Git and cleanup features
2026-02-10 16:10:56 +08:00
f41fb6167d
feat(kb-workflow): Add Git version control and cleanup
...
## Git Version Management
- Git repository in archive/ directory
- Environment variables for configuration (KB_GIT_ROOT, KB_GIT_REMOTE, etc.)
- Auto commit after archive with retry (3 attempts)
- Status command shows git status
## Subscription Management
- Subscriptions stored in archive/subscriptions/
- Track processed items to avoid duplicates
- Git tracked for history
## Automatic Cleanup
- Cleanup strategy based on file type and age
- video/frame retention: 7 days
- metadata/reports: permanent
- archive/: Git managed
## New Commands
- --trigger git-init: Initialize git repository
- --trigger git-status: Show git status
- --trigger cleanup: Clean expired temporary files
2026-02-10 16:09:50 +08:00
8b43416ab7
feat(kb-deep-analyzer): Enhance template and metadata
...
- Add complete metadata to frontmatter (bvid, cvid, mid, platform)
- Add AI-generated tags in analysis prompt
- Update templates to include full metadata
- Add tags_string for YAML array format
2026-02-10 15:42:09 +08:00
f2309bf2d1
feat(kb-workflow): Add complete file storage structure
...
- Define clear file storage rules:
- content/downloads/ for video files
- content/subtitles/ for subtitle files
- frames/ for video frames
- Track all created files in process results
- Add directory structure documentation
- Show file counts and total size in status
2026-02-10 15:28:41 +08:00
3f74bcd377
docs: Add subtitle enhancement tasks to TODO
...
- Add visual model subtitle extraction (GPT-4V/Claude/GLM-4V)
- Add ASR subtitle extraction (Whisper/讯飞/阿里云)
2026-02-10 15:16:47 +08:00
f4c6612849
docs: Add workspace configuration and memory files
...
- AGENTS.md: Update agent workspace guidelines (read TODO.md in workflow)
- TODO.md: Add security inspection and TTS configuration tasks
- memory/: Add knowledge base refactoring progress notes
- .gitignore: Add debug/ directory
2026-02-10 14:56:18 +08:00
47355b36d9
feat(kb-bilibili-content): Add article content extraction using Playwright
...
- Replace API-based article extraction with Playwright scraping
- Support Bilibili article (专栏) content extraction
- Extract full article text using .read-article-holder selector
- Use async Playwright API for proper async/await support
2026-02-10 14:53:05 +08:00
56907e31f5
refactor(kb-zhihu-metadata): Simplify to only support zvideo and zhuanlan
...
- Remove question, answer, user types (not needed for knowledge base)
- Keep only zvideo (知乎视频) and zhuanlan (知乎专栏文章)
- Update URL patterns and extraction logic
- Simplified CLI interface
2026-02-10 14:46:39 +08:00
d216ce4ba7
feat(kb-bilibili-metadata): Add UP subscription tracking support
...
- Enhance _extract_up_async() with subscription tracking:
- video_count: Number of videos
- article_count: Number of articles
- subscription_info: Last video/article for update detection
- Update CLI to support UP subscription queries
- All collection types now support basic_info + subscription status
2026-02-10 14:34:12 +08:00
26accdd726
fix(kb-bilibili-content): Improve browser headers and cookie support
...
- Add complete browser headers for better API compatibility
- Read BILIBILI_COOKIE from environment variable
- Add debug print statements for troubleshooting
2026-02-10 14:29:35 +08:00
4cdbc825ce
feat(kb-bilibili-metadata): Add series/readlist/favlist metadata extraction
...
- Add extract_series_async() for series collection metadata
- Add extract_readlist_async() for article list metadata
- Add extract_favlist_async() for favorite list metadata
- Add CLI support for new types: series, readlist, favlist
- Update bilibili_metadata.py with new extraction functions
- Add --mid parameter for series type
Note: These types record basic info and subscription status only,
not the full content list (which may be large).
2026-02-10 14:27:14 +08:00
51347f671b
feat: Improve kb-bilibili-content video download
...
- Simplify download logic using playurl API
- Support video download without Cookie (FLV format)
- Improve subtitle extraction
- Better error handling and progress display
- Remove unnecessary dependencies
2026-02-10 13:40:03 +08:00
a7ab478dbc
fix: Improve kb-bilibili-metadata output format
...
- Wrap result in standard format with success/data/metadata
- Change default headless to False (use API by default)
- Add error handling with structured output
- Improve CLI output readability
2026-02-10 13:18:58 +08:00
066395164a
feat: Add Xiaohongshu (小红书) URL support
...
- Add xhslink.com short link resolution
- Add xiaohongshu.com note/explore/notes URL patterns
- Add discovery/item pattern (resolved from short links)
- Add user profile pattern
- Support multiple Xiaohongshu URL formats
2026-02-10 13:00:38 +08:00
c14575ed1f
feat: Add comprehensive Zhihu URL support
...
- Add zhuanlan article support (zhuanlan.zhihu.com/p/{id})
- Add organization page support (zhihu.com/org/{name})
- Add organization subpages: answers, zvideos, asks, posts, pins
- Add zvideo support (zhihu.com/zvideo/{id})
- Improve URL pattern matching for various Zhihu content types
2026-02-10 12:51:38 +08:00
3fb5af33ab
feat: Add mobile opus URL support for Bilibili
...
- Support m.bilibili.com/opus/{id} for dynamic posts
- Handle mobile URLs when resolving short links
2026-02-10 12:38:24 +08:00
a3623b5338
feat: Add mobile URL support for Bilibili
...
- Support m.bilibili.com/space/{id} for UP主 accounts
- Handle mobile video URLs when resolving short links
2026-02-10 12:35:06 +08:00
d72090b957
feat: Add automatic short link resolution for b23.tv
...
- Use requests library to follow redirects and resolve b23.tv short links
- Automatically extract BV ID from resolved URL
- Support both short links and full URLs seamlessly
2026-02-10 12:33:34 +08:00
115d6e042e
fix: Improve kb-link-parser URL patterns
...
- Add playlist pattern for bilibili series (lists/xxx?type=series)
- Add favlist pattern for bilibili favorites (favlist?fid=xxx)
- Add readlist pattern for bilibili article collections (read/readlist/rlxxx)
- Reorder pattern matching to check specific patterns before generic space pattern
2026-02-10 12:30:55 +08:00
5507b1d986
feat: Add KB knowledge base series skills
...
- kb-link-parser: Multi-platform URL parser (Bilibili, YouTube, Zhihu)
- kb-bilibili-metadata: Bilibili video/article/up主 metadata extractor
- kb-zhihu-metadata: Zhihu question/answer/user metadata extractor
- kb-bilibili-content: Bilibili video download and content extraction
- kb-ocr-processor: OCR text extraction from video frames
- kb-deep-analyzer: AI-powered deep content analysis
- kb-workflow: Workflow orchestration (inbox, archive, subscriptions, report)
Also add:
- ARCHITECTURE_FLOWCHART.md: System architecture diagrams
- DETAILED_DESIGN.md: Complete design documentation
2026-02-10 12:18:52 +08:00
2f5a269866
feat(bilibili-analyzer): 支持字幕区域裁剪
...
- 新增 --crop-ratio 参数，提取底部1/N区域（默认1/4）
- 新增 --no-crop 参数，禁用裁剪提取完整帧
- 优化字幕识别效率，只处理字幕区域
2026-02-09 23:51:42 +08:00
732dfb6549
清理GitHub相关技能
2026-02-09 23:37:58 +08:00
79381bf0ba
移除不需要的AI/开发技能
2026-02-09 23:25:01 +08:00
59d873e1ad
清理workspace：删除临时文件、测试脚本和项目文档
2026-02-09 23:19:15 +08:00
6051d05f0c
修改-analyzer技能
2026-02-09 22:21:51 +08:00
e9872afd25
feat: 更新子代理监督规则和文件存放规则
...
- 新增子代理监督规则（子代理必须严格遵守技能要求，违规打回重做）
- 新增文件存放规则（默认输出到~/work/）
- 将MEMORY.md中的规则同步到AGENTS.md
2026-02-09 20:24:56 +08:00
00e55259a7
Update: workspace changes
2026-02-09 18:43:33 +08:00
e0a2b323d9
Update obsidian-kb skill and add find-skills
...
- obsidian-kb: Add OCR processors (easyocr, tesseract)
- obsidian-kb: Fix subtitle detection with test files
- obsidian-kb: Add cookie verification script
- Add find-skills skill for discovering capabilities
- Add video processing scripts
- Update clawhub lock file
2026-02-09 11:53:42 +08:00
d66eade77e
docs: 更新实施计划 - 所有 Checkpoints 已完成
...
- 所有核心模块已实现
- 集成主流程完成
- 配置和文档已更新
- 代码已提交推送
项目状态：✅ 已完成所有 Checkpoints (1-6)
测试状态：⏳ 待实际视频验证
下一步：可在生产环境中直接使用硬字幕提取功能
2026-02-09 00:39:00 +08:00
1e96b8785c
feat(hard-subtitle): 集成硬字幕提取功能
...
- 添加 VideoStreamExtractor - B站 API + OpenCV 流式读取
- 添加 SubtitleDetector - MSER 检测 + 颜色过滤 + LRU 缓存
- 添加 OCRProcessor - PaddleOCR 集成 + 批量识别 + 过滤去重
- 添加 HardSubtitleExtractor - 主协调器，整合所有模块
- 更新 BilibiliExtractor - 优先官方字幕，降级硬字幕提取
- 更新 __init__.py - 导出所有新模块
- 添加配置项支持
- 完成完整实施计划 Checkpoint 5
功能：
- 支持官方 CC 字幕（WBI API）
- 支持硬字幕提取（PaddleOCR）
- 自动检测字幕区域
- 字幕结果缓存
- 去重和噪声过滤
2026-02-09 00:37:16 +08:00
cd80837c86
fix(bilibili): 改进字幕提取，支持 WBI API 和多语言优先级
...
- 使用 WBI API 获取字幕信息（支持 AI 字幕）
- 优先选择中文字幕（ai-zh > zh-CN > zh-Hans）
- 添加硬字幕提取实施计划
- 改进错误处理，静默失败不影响主流程
2026-02-09 00:24:19 +08:00
f952c963d0
添加知乎 DrissionPage 计划和 Playwright 完成报告
2026-02-08 05:57:53 +08:00
1737800a3c
清理临时测试文件和报告
2026-02-08 05:57:40 +08:00
30eb8489a2
添加知乎 DrissionPage 完成报告和更新 .gitignore
...
- 添加最终完成报告到 memory/completion-reports/
- 更新 .gitignore 排除测试文件和临时报告
2026-02-08 05:57:22 +08:00
a69225ff2b
知乎提取器：从 Playwright 迁移到 DrissionPage 方案
...
- 使用 DrissionPage 替代 Playwright
- 实现 Cookie 管理和反爬绕过
- 支持回答/文章/问题三种内容类型
- 灵活配置（文件/环境变量/配置文件）
- 更新文档和配置文件
2026-02-08 05:56:27 +08:00
aa206a1679
Add knowledge base: B站视频《在2000多米高空 建座"会飞的"发电站》
...
- 新增 knowledge-base/ 目录
- 保存央视新闻视频：在2000多米高空 建座"会飞的"发电站
- 完善 obsidian-kb skill：添加知乎 Playwright 提取器
- 更新依赖和文档
2026-02-07 19:06:38 +08:00
027b82eb28
重构 B站提取器：移除技能配置，添加Python脚本实现
2026-02-07 17:27:23 +08:00
e4c5dbf4b7
修改obsidian-kb技能描述：改为明确触发模式，不自动触发
2026-02-07 15:50:09 +08:00
c79035d6ff
修复B站短链接解析问题和处理流程说明
...
1. 修复短链接解析bug：
- 新增 _resolve_shortlink_bvid() 方法
- 使用 Playwright + stealth 模式解析 b23.tv 短链接
- 确保反爬机制正常工作
2. 更新 SKILL.md 使用说明：
- 添加明确的处理流程说明
- 说明收到链接后直接处理，不要问用户
- 添加示例触发词
修复问题：短链接无法自动解析，反爬机制未生效
2026-02-07 15:31:47 +08:00
8163c37e83
refactor: 移除bilibili_extractor插件，改用obsidian-kb
...
- 删除 bilibili_extractor.py 及相关文档和配置
- 删除 bilibili_venv 虚拟环境
- 删除测试脚本和生成的B站笔记
- 以后使用 skills/obsidian-kb 提供的B站提取功能
2026-02-07 14:44:58 +08:00
f15c494d37
feat: 添加B站登录配置支持和视频笔记
...
- bilibili_extractor.py: 添加SESSDATA登录凭证支持
- 新增 bilibili_config.example 配置模板
- 更新 .gitignore: 排除虚拟环境和敏感配置文件
- 添加今日记忆 (2026-02-07.md)
- 新增B站视频笔记: Qwen3-TTS上新、时代真变了、用了几天Clawdbot
2026-02-07 14:38:19 +08:00
08f7c9f26c
feat: integrate bilibili-extractor into obsidian-kb
...
- Add extractors layer with BilibiliExtractor (Playwright stealth mode)
- Add outputs layer with ObsidianWriter and GitSync
- Implement subtitle/no-subtitle graceful degradation
- Update config with output and bilibili extraction settings
- Complete integration pipeline: extract → process → output → sync
- Add comprehensive tests (100% pass rate)
- Update documentation (SKILL.md, README.md, requirements.txt)
2026-02-07 13:07:43 +08:00
ceeeb0f84c
添加bilibili-extractor技能
...
创建了独立的skill目录和SKILL.md文档：
- 完整的功能说明和使用方法
- 新会话可通过技能系统使用B站视频提取功能
- 包含故障排除、技术栈、依赖说明等
技能位置: skills/bilibili-extractor/
2026-02-07 09:54:19 +08:00
87ba038d39
添加B站视频内容提取工具（反爬虫方案）
...
功能特性:
- 使用Playwright + undetected-playwright绕过B站极验验证码
- 支持提取视频基本信息、统计数据、字幕
- 生成结构化Markdown笔记
- 多策略数据提取（JS变量→HTML解析→API备用）
- 完善的错误处理和超时机制
文件说明:
- bilibili_extractor.py: 主程序脚本（21KB）
- BILIBILI_EXTRACTOR_README.md: 完整技术文档（6.0KB）
- USAGE.md: 快速使用指南（1.5KB）
- PROJECT_SUMMARY.md: 项目总结（2.9KB）
- notes/: 生成的视频笔记目录
- .gitignore: 忽略Python虚拟环境等
测试验证:
- 成功提取BV11t64BJEDZ视频内容
- 绕过B站极验验证码
- 生成完整的Markdown笔记
2026-02-07 09:51:22 +08:00
f41f945fe0
添加B站链接转换和浏览器安装脚本
...
- bilibili-short-to-long.sh: B站短链接转长链接工具
- 支持短链接、BV号、AV号多种格式
- 自动清理URL中的tracking参数
- 友好的输出格式
- install-browser-for-openclaw.sh: OpenClaw浏览器环境一键安装脚本
- 支持Ubuntu/Debian/CentOS
- 自动安装Chromium浏览器
- 配置OpenClaw Gateway
2026-02-07 09:07:42 +08:00
7032ddb1a3
feat(ai-client): 实现多 AI 提供商支持与 fallback 机制
...
- 支持 MiniMax、智谱、通义千问等多家 AI 提供商
- 实现自动 fallback 机制（主失败则自动切换备用）
- 修复 MiniMax Anthropic 格式响应解析
- 修复智谱 API 端点（自动添加 /chat/completions）
- 新增 _call_api_with_fallback 方法统一处理 fallback 逻辑
- 更新 _load_ai_config 加载主配置和 fallbacks 配置
配置文件位置: ~/.config/obsidian-kb/config.yaml
2026-02-07 07:08:46 +08:00
OpenClaw Agent