docs: map existing codebase

This commit is contained in:
2026-03-13 14:55:43 +08:00
parent 1d7957aab4
commit a8669ec0a3
5 changed files with 790 additions and 0 deletions
+154
View File
@@ -0,0 +1,154 @@
# Architecture
**Analysis Date:** 2026-03-13
## Pattern Overview
**Overall:** Multi-skill workflow orchestration with skill-based modular architecture
**Key Characteristics:**
- Skill-based modular design: Each skill is an independent module with its own `SKILL.md` specification
- Pipeline workflow pattern: Sequential processing stages orchestrated by `kb-workflow`
- Stateful session management: tmux-based persistent sessions for GSD-OpenClaw
- Platform abstraction: Unified interfaces for multiple content platforms (Bilibili, Zhihu, YouTube, etc.)
## Layers
**Skill Definition Layer:**
- Purpose: Define skill capabilities, triggers, and execution steps
- Location: `kb-*/SKILL.md`, `gsd-openclaw/skill/SKILL.md`
- Contains: Markdown documents with YAML frontmatter and structured workflows
- Depends on: Nothing (declarative)
- Used by: AI agents that execute skills
**Core Implementation Layer:**
- Purpose: Implement actual functionality for each skill
- Location: `kb-*/*.py`, `gsd-openclaw/src/gsd_openclaw/*.py`
- Contains: Python modules with CLI entry points and core logic
- Depends on: External APIs, platform SDKs, AI providers
- Used by: Skill orchestration layer
**Orchestration Layer:**
- Purpose: Coordinate multiple skills in workflows
- Location: `kb-workflow/kb_workflow.py`
- Contains: Workflow coordination utilities, directory management, Git integration
- Depends on: All skill modules
- Used by: Main agent executing workflows
**Session Management Layer:**
- Purpose: Manage persistent tmux sessions for long-running operations
- Location: `gsd-openclaw/src/gsd_openclaw/tmux_session.py`
- Contains: TmuxSessionManager, session lifecycle management
- Depends on: libtmux, StateManager, WorkspaceManager
- Used by: GSD command orchestration
**State Persistence Layer:**
- Purpose: Provide crash-safe state checkpointing
- Location: `gsd-openclaw/src/gsd_openclaw/state_manager.py`
- Contains: StateManager, atomic save/load with backup rotation
- Depends on: File system only
- Used by: TmuxSessionManager, workspace management
## Data Flow
**Content Processing Pipeline:**
1. URL Input → `kb-link-parser` extracts platform, type, and ID
2. Metadata Extraction → Platform-specific skill (`kb-*-metadata`) fetches metadata
3. Content Download → Platform-specific skill (`kb-*-content`) downloads content
4. Speech Recognition → `kb-whisper`/`kb-funasr` transcribes audio (if needed)
5. Deep Analysis → `kb-deep-analyzer` generates AI-powered analysis report
6. Archive → `kb-workflow` archives to Git-managed directory
**GSD Session Flow:**
1. `/gsd:*` command received
2. `TmuxSessionManager.create_session()` or `resume_session()`
3. State persisted via `StateManager.save()`
4. Command executed in tmux session
5. Output captured and formatted via `FeishuResponseFormatter`
**State Management:**
- State stored in `~/gsd-workspaces/{project}/.gsd/state.json`
- Atomic writes via temp file + rename
- Automatic backup rotation (state.json → state.json.bak)
## Key Abstractions
**Skill:**
- Purpose: Represents an independent capability unit
- Examples: `kb-link-parser/link-parser.py`, `kb-deep-analyzer/deep_analyzer.py`
- Pattern: CLI-based with JSON output, defined by `SKILL.md` specification
**TmuxSession:**
- Purpose: Represents a managed tmux session for project isolation
- Examples: `gsd-openclaw/src/gsd_openclaw/tmux_session.py`
- Pattern: Dataclass with session_id, project_name, tmux_session_name
**Workspace:**
- Purpose: Isolated directory structure per project
- Examples: `gsd-openclaw/src/gsd_openclaw/workspace.py`
- Pattern: Defined subdirectories (.gsd/, code/, io-logs/)
**AnalysisResult:**
- Purpose: Structured output from deep analysis
- Examples: `kb-deep-analyzer/deep_analyzer.py`
- Pattern: Dataclass with summary, main_topics, key_insights, tags
## Entry Points
**CLI Entry Points:**
- Location: `kb-*/*.py` with `if __name__ == '__main__': main()`
- Triggers: Command-line invocation with argparse
- Responsibilities: Parse arguments, invoke core logic, output JSON
**gsd-ocl CLI:**
- Location: `gsd-openclaw/src/gsd_openclaw/cli.py`
- Triggers: `gsd-ocl create|list|destroy|resume|cleanup`
- Responsibilities: Session lifecycle management
**Skill Trigger:**
- Location: `SKILL.md` frontmatter with `name` and `description`
- Triggers: AI agent pattern matching on user messages
- Responsibilities: Define when and how to invoke the skill
## Error Handling
**Strategy:** Fail-fast with structured JSON error output
**Patterns:**
- All CLI tools output JSON with `success: true/false` field
- Error responses include `error` and `detail` fields
- Deep analysis failure terminates entire workflow (per `kb-workflow/SKILL.md`)
- State manager uses backup fallback on corruption
**Error Response Format:**
```json
{
"success": false,
"error": "Error description",
"detail": "Additional context"
}
```
## Cross-Cutting Concerns
**Logging:** Console output with `[ModuleName]` prefixes (e.g., `[DeepAnalyzer]`, `[AI Client]`)
**Validation:**
- Environment variable checks before workflow execution (`kb-workflow/SKILL.md`)
- Required field validation in `StateManager.load()`
- Platform/type validation in link parser
**Authentication:**
- Platform cookies via environment variables (`BILIBILI_COOKIE`, `ZHIHU_COOKIE`)
- AI API keys via environment variables (`AI_API_KEY`, `HF_TOKEN`)
- Git credentials via environment variables (`KB_GIT_USER`, `KB_GIT_EMAIL`)
**Configuration:**
- Environment variables loaded from `~/.kb_env`
- Path configuration via environment variables (`KB_BASE_PATH`, `KB_ARCHIVE_PATH`, `KB_GIT_ROOT`)
---
*Architecture analysis: 2026-03-13*
+180
View File
@@ -0,0 +1,180 @@
# Codebase Concerns
**Analysis Date:** 2026-03-13
## Tech Debt
### 异常处理不完善
- Issue: 多处使用 bare `except:` 或 `except Exception` 吞掉所有异常,没有记录错误信息
- Files:
- `kb-bilibili-metadata/bilibili_extractor.py` (第 310、340 行)
- `kb-deep-analyzer/ai_client.py` (第 228、489 行)
- `kb-deep-analyzer/deep_analyzer.py` (第 394、410、506、516 行)
- `kb-whisper-diarization/whisper_diarization.py` (第 70、326 行)
- `kb-funasr/funasr_tool.py` (第 81、186 行)
- Impact: 错误被静默忽略,调试困难,生产问题难以追踪
- Fix approach: 使用具体异常类型,添加日志记录
### 过多的 return None 模式
- Issue: 多个函数在异常或边界情况返回 `None`,调用方需要额外检查
- Files:
- `kb-bilibili-metadata/bilibili_extractor.py` (第 140、161、176 行)
- `kb-bilibili-content/video_downloader.py` (第 36、48、68、75、90、106、148 行)
- `kb-link-parser/link-parser.py` (第 197、245、379、436 行)
- `kb-link-parser/link_parser_shortlink.py` (第 223、237、249 行)
- `kb-zhihu-metadata/zhihu_extractor.py` (第 137、152、410、437、450 行)
- Impact: 调用方需要频繁检查 None,代码冗长,容易遗漏
- Fix approach: 使用 Result 模式或抛出具体异常
### 硬编码的默认路径
- Issue: 默认知识库路径硬编码在代码中
- Files: `kb-workflow/kb_workflow.py` (第 104、113、122 行)
- Impact: 不同用户或环境需要修改代码或设置环境变量
- Fix approach: 使用配置文件或更灵活的路径发现机制
### AI 客户端重复实现
- Issue: 存在两套 AI 客户端实现 (`ai_client.py` 和 `deep_analyzer.py` 中的 `SimpleAIClient`)
- Files:
- `kb-deep-analyzer/ai_client.py` (514 行,功能完整)
- `kb-deep-analyzer/deep_analyzer.py` (37-123 行,简化版本)
- Impact: 代码重复,维护困难,行为可能不一致
- Fix approach: 统一使用一个 AI 客户端实现
## Known Bugs
### CI/CD 部署列表与实际目录不匹配
- Symptoms: `.gitea/workflows/deploy.yml` 中的技能列表包含 `dev-workflow`,但该目录已被删除(根据 git log)
- Files: `.gitea/workflows/deploy.yml` (第 23 行)
- Trigger: 推送到 main 分支时,CI 会尝试同步不存在的目录
- Workaround: 目录不存在时 rsync 会跳过,但会产生混乱的日志
### Playwright 依赖检测不完整
- Symptoms: 多个模块检测 Playwright 可用性但不处理所有边界情况
- Files: `kb-bilibili-metadata/bilibili_extractor.py` (第 28-44 行)
- Trigger: 当 Playwright 不可用时回退到 API,但 httpx 也可能不可用
- Workaround: 检查 HTTPX_AVAILABLE 并返回错误信息
## Security Considerations
### 环境变量中传递敏感信息
- Risk: API Keys 和 Cookies 通过环境变量传递,可能在日志或错误信息中泄露
- Files:
- `kb-deep-analyzer/deep_analyzer.py` (第 49-52 行直接读取环境变量)
- `kb-deep-analyzer/ai_client.py` (第 32、44 行)
- `kb-zhihu-metadata/zhihu_extractor.py` (Cookie 处理)
- Current mitigation: 使用 `.env.example` 文档说明,但无运行时保护
- Recommendations:
- 敏感信息不应出现在日志输出中
- 考虑使用 secrets 管理工具
### Git 凭据在代码中配置
- Risk: `kb_workflow.py` 中 Git 配置通过环境变量传递用户名和邮箱
- Files: `kb-workflow/kb_workflow.py` (第 196-200 行)
- Current mitigation: 环境变量方式
- Recommendations: 考虑使用 Git credential helper
## Performance Bottlenecks
### 固定的 HTTP 超时
- Problem: 多处 HTTP 请求使用固定的 30 秒超时
- Files:
- `kb-deep-analyzer/ai_client.py` (第 135、169、191、213 行)
- `kb-bilibili-metadata/bilibili_extractor.py` (第 262 行)
- Cause: 大文件下载或慢速 API 可能超时
- Improvement path: 根据请求类型使用可配置的超时
### AI 调用长超时
- Problem: AI API 调用使用 600 秒(10 分钟)超时
- Files: `kb-deep-analyzer/deep_analyzer.py` (第 87、97、116 行)
- Cause: 长文本分析需要时间,但 10 分钟过长
- Improvement path: 根据内容长度动态调整超时
### Playwright 浏览器启动开销
- Problem: 每次提取都启动新的浏览器实例
- Files:
- `kb-bilibili-metadata/bilibili_extractor.py`
- `kb-zhihu-metadata/zhihu_extractor.py`
- Cause: 无浏览器实例复用
- Improvement path: 考虑浏览器实例池或持久化
## Fragile Areas
### 网页选择器依赖
- Files:
- `kb-zhihu-metadata/zhihu_extractor.py` (第 183-327 行)
- `kb-bilibili-metadata/bilibili_extractor.py`
- Why fragile: 网站更新会改变 CSS 类名和 DOM 结构
- Safe modification: 添加选择器配置化,使用多个备用选择器
- Test coverage: 缺少针对选择器变更的测试
### 元数据格式多样性
- Files: `kb-deep-analyzer/deep_analyzer.py` (第 382-414 行 `_normalize_metadata`)
- Why fragile: 不同来源的元数据格式不一致(扁平、嵌套、混合)
- Safe modification: 使用数据验证库(如 Pydantic)标准化格式
- Test coverage: 部分覆盖
### Tmux 会话状态管理
- Files: `gsd-openclaw/src/gsd_openclaw/tmux_session.py`
- Why fragile: tmux 会话可能被外部操作破坏,状态文件可能不同步
- Safe modification: 添加状态校验和恢复机制
- Test coverage: 有测试 (`test_tmux_session.py`),但边缘情况可能不足
## Scaling Limits
### 单实例设计
- Current capacity: 单用户、单进程
- Limit: 无法并行处理多个分析任务
- Scaling path: 引入任务队列和 worker 进程
### 内存中的用户上下文缓存
- Current capacity: 无限(但无清理机制)
- Limit: 长时间运行后内存泄漏
- Files: `gsd-openclaw/src/gsd_openclaw/orchestrator.py` (第 70 行 `user_context_managers` 字典)
- Scaling path: 添加 LRU 缓存或定期清理
## Dependencies at Risk
### 外部 API 依赖
- Risk: 依赖 Bilibili、知乎等第三方 API,可能随时变更
- Impact: 抓取功能失效
- Migration plan: 使用官方 SDK(如果有),或添加 API 版本检测
### Playwright 和 undetected-playwright
- Risk: `undetected_playwright` 是小众库,维护可能不活跃
- Impact: 反爬虫规避功能失效
- Files: `kb-bilibili-metadata/bilibili_extractor.py` (第 35-38 行)
- Migration plan: 监控库更新,准备降级方案
## Missing Critical Features
### 缺少统一日志
- Problem: 各模块使用 `print()` 输出日志,无统一日志级别控制
- Blocks: 生产环境日志管理、问题排查
### 缺少健康检查端点
- Problem: 无服务健康状态监控
- Blocks: 运维监控、自动故障恢复
## Test Coverage Gaps
### kb-* 技能缺少单元测试
- What's not tested: 所有 kb-* 技能目录(kb-bilibili-content, kb-zhihu-metadata 等)都没有测试文件
- Files: 所有 `kb-*/**/*.py` 文件
- Risk: 重构或修改可能引入回归 bug
- Priority: High - 核心功能缺少测试保护
### 集成测试缺失
- What's not tested: 技能之间的集成(如 kb-workflow 调用其他技能)
- Files: 跨技能调用路径
- Risk: 接口变更导致集成失败
- Priority: Medium
### 边缘情况测试不足
- What's not tested: 网络错误、超时、无效输入等
- Files: `gsd-openclaw/tests/` 中的测试主要覆盖正常路径
- Risk: 生产环境异常处理不当
- Priority: Medium
---
*Concerns audit: 2026-03-13*
+162
View File
@@ -0,0 +1,162 @@
# External Integrations
**Analysis Date:** 2026-03-13
## APIs & External Services
**AI Providers:**
- **智谱 AI (Zhipu)** - 主要 AI 分析服务
- SDK: `requests` HTTP 客户端
- Auth: `AI_API_KEY` 环境变量
- Base URL: `https://open.bigmodel.cn/api/paas/v4`
- 模型: `glm-5`, `glm-4`
- **MiniMax** - AI 备选服务
- SDK: `requests` HTTP 客户端
- Auth: `AI_API_KEY` 环境变量
- Base URL: `https://api.minimaxi.com/anthropic/v1/messages`
- **OpenAI** - AI 备选服务
- SDK: `requests` HTTP 客户端
- Auth: `AI_API_KEY` 环境变量
- **阿里云通义千问 (Alibaba/Qwen)** - AI 备选服务
- SDK: `requests` HTTP 客户端
- Auth: `AI_API_KEY` 环境变量
**视频平台 API:**
- **Bilibili** - 视频/专栏元数据提取
- SDK: `httpx`, `playwright`
- Auth: `BILIBILI_COOKIE` 环境变量 (SESSDATA, bili_jct)
- 文件: `kb-bilibili-metadata/bilibili_extractor.py`
- **知乎** - 问答/专栏/视频内容
- SDK: `yt-dlp`, `playwright`
- Auth: `ZHIHU_COOKIE` 环境变量
- 文件: `kb-zhihu-metadata/zhihu_metadata.py`, `kb-zhihu-content/zhihu_content.py`
- **YouTube** - 视频内容(通过链接解析)
- SDK: `yt-dlp`
- 文件: `kb-link-parser/link-parser.py`
- **小红书** - 笔记内容
- 短链接: `xhslink.com`
- 文件: `kb-link-parser/link-parser.py`
## Data Storage
**Databases:**
- 无数据库 - 所有数据以文件形式存储
**File Storage:**
- 本地文件系统
- 工作目录: `~/work/knowledge-base/`
- 结构:
- `pending/` - 待处理链接
- `metadata/` - 元数据 JSON
- `content/downloads/` - 下载的媒体文件
- `analyzed/` - AI 分析报告
**归档存储:**
- 路径: `~/work/zq/30.资源/AI分析/`
- 按日期组织: `YYYY-MM-DD/`
**Caching:**
- 模型缓存: `~/.cache/modelscope/hub/` - FunASR 模型
## Authentication & Identity
**Auth Provider:**
- Cookie-based 认证
- Bilibili: `BILIBILI_COOKIE`
- 知乎: `ZHIHU_COOKIE`
**AI API 认证:**
- API Key 认证
- `AI_API_KEY` - AI 服务密钥
- `HF_TOKEN` - HuggingFace Token (说话者分离)
## Monitoring & Observability
**Error Tracking:**
- 无外部监控服务
**Logs:**
- 控制台输出 - 所有技能通过 `print()` 输出日志
- 建议使用 `python3 -u` 禁用缓冲以实时查看输出
## CI/CD & Deployment
**Hosting:**
- Self-hosted (本地服务器)
- 技能部署位置: `/home/starmen1265/.openclaw/skills/`
**CI Pipeline:**
- Gitea Actions (self-hosted runner)
- 配置文件: `.gitea/workflows/deploy.yml`
- 触发: push to main
- 使用 `rsync` 同步技能目录
## Environment Configuration
**必须的环境变量:**
```bash
AI_API_KEY="your-api-key" # AI 服务密钥
AI_PROVIDER="zhipu" # 或 openai, minimax, alibaba
AI_MODEL="glm-5" # 模型名称
AI_BASE_URL="https://..." # API 地址
```
**可选的环境变量:**
```bash
BILIBILI_COOKIE="SESSDATA=xxx; bili_jct=xxx" # B站登录态
ZHIHU_COOKIE="..." # 知乎登录态
HF_TOKEN="hf_xxx" # HuggingFace Token
KB_GIT_USER="Your Name" # Git 提交用户名
KB_GIT_EMAIL="your@email.com" # Git 提交邮箱
KB_ARCHIVE_PATH="~/work/zq/30.资源/AI分析" # 归档目录
KB_GIT_ROOT="~/work/zq" # Git 仓库根目录
KB_BASE_PATH="~/work/knowledge-base" # 知识库基础路径
WHISPER_CPP_PATH="~/whisper.cpp" # Whisper.cpp 路径
```
**配置文件位置:**
- `~/.kb_env` - 知识库专用配置(推荐)
- `kb-workflow/.env.example` - 示例配置
**Secrets location:**
- 环境变量(明文存储在 `~/.kb_env`)
- 无加密/secrets 管理
## Webhooks & Callbacks
**Incoming:**
- 无 Webhook 端点
**Outgoing:**
- 无主动 Webhook 调用
## External Tools
**音视频处理:**
- `ffmpeg` / `ffprobe` - 音视频格式转换和元数据提取
- `yt-dlp` - 视频下载(知乎视频)
**语音识别:**
- `whisper.cpp` - 本地 Whisper 推理
- `funasr` / `modelscope` - 阿里 FunASR
**浏览器自动化:**
- `playwright` - 无头浏览器
- `undetected_playwright` - 反检测浏览器
**OCR:**
- `tesseract` - OCR 引擎
- `easyocr` - 深度学习 OCR
**会话管理:**
- `tmux` - 终端会话管理(gsd-openclaw)
---
*Integration audit: 2026-03-13*
+108
View File
@@ -0,0 +1,108 @@
# Technology Stack
**Analysis Date:** 2026-03-13
## Languages
**Primary:**
- Python 3.12 - 主要用于 kb-* 技能模块(内容采集、语音识别、AI分析等)
- JavaScript/Node.js >=16.7.0 - 用于 gsd (Get-Shit-Done) 元提示系统
**Secondary:**
- Shell/Bash - 用于部署脚本和工具链(ffmpeg, yt-dlp)
## Runtime
**Environment:**
- Python 3.12.3
- Node.js v22.22.0
**Package Manager:**
- pip/uv (Python) - gsd-openclaw 使用 uv.lock
- npm (Node.js) - gsd 使用 npm
**Lockfile:**
- `uv.lock` (gsd-openclaw)
- `package-lock.json` (gsd)
## Frameworks
**Core:**
- 无传统 Web 框架 - 这是一个 CLI 技能集合,非 Web 应用
**Testing:**
- pytest >=7.0.0 - Python 单元测试 (gsd-openclaw)
- Node.js 内置测试 - gsd 使用自定义 `scripts/run-tests.cjs`
**Build/Dev:**
- esbuild ^0.24.0 - gsd hooks 构建
- c8 ^11.0.0 - 测试覆盖率工具
## Key Dependencies
**Python 核心依赖:**
- `requests` - HTTP 客户端,用于 AI API 调用
- `httpx` - 异步 HTTP 客户端(Bilibili 元数据提取)
- `playwright` / `undetected_playwright` - 浏览器自动化(反爬虫)
- `easyocr>=1.7.0` - OCR 文字识别
- `pytesseract>=0.3.10` - Tesseract OCR 接口
- `Pillow>=9.0.0` - 图像处理
- `torch>=2.0.0` - PyTorch 深度学习框架
- `pyannote.audio>=1.0.0` - 说话者分离
- `funasr` / `modelscope` - 阿里 FunASR 语音识别
- `libtmux>=0.55.0` - tmux 会话管理 (gsd-openclaw)
**Node.js 核心依赖:**
- 无外部运行时依赖 - gsd 是纯 CommonJS 工具集
## Configuration
**环境变量配置:**
- 配置文件: `~/.kb_env` - 知识库专用环境变量
- `.env.example` - 各技能目录下提供示例配置
**关键配置文件:**
- `pyproject.toml` - gsd-openclaw 项目配置
- `requirements.txt` - Python 技能依赖声明
- `package.json` - gsd Node.js 项目配置
**构建配置:**
- `scripts/build-hooks.js` - gsd hooks 构建
- `tool.setuptools` - gsd-openclaw 打包配置
## Platform Requirements
**Development:**
- ffmpeg - 音视频格式转换
- yt-dlp - 视频下载工具
- whisper.cpp - 本地语音识别(或 GPU 版 PyTorch)
- tesseract - OCR 引擎(可选)
- tmux - 会话管理
**Production:**
- 自托管 Gitea Actions runner
- 部署目标: `~/.openclaw/skills/` 目录
- rsync 同步部署
## Project Structure
**Monorepo 结构:**
```
kb-skills/
├── gsd/ # Get-Shit-Done 元提示系统 (Node.js)
├── gsd-openclaw/ # GSD-OpenClaw 桥接技能 (Python)
├── kb-workflow/ # 知识库工作流编排入口
├── kb-link-parser/ # 多平台 URL 解析
├── kb-bilibili-*/ # B站内容采集技能
├── kb-zhihu-*/ # 知乎内容采集技能
├── kb-whisper/ # Whisper 语音识别
├── kb-whisper-diarization/ # 说话者分离
├── kb-funasr/ # 阿里 FunASR 语音识别
├── kb-ocr-processor/ # OCR 文字提取
├── kb-deep-analyzer/ # AI 深度内容分析
└── todo-manager/ # 待办管理技能
```
---
*Stack analysis: 2026-03-13*
+186
View File
@@ -0,0 +1,186 @@
# Codebase Structure
**Analysis Date:** 2026-03-13
## Directory Layout
```
/home/starmen1265/work/kb-skills/
├── .claude/ # Claude Code configuration
├── .gitea/ # Gitea CI/CD workflows
├── .planning/ # Planning documents
│ ├── codebase/ # Codebase analysis docs
│ └── research/ # Research documents
├── gsd/ # GSD (Get Shit Done) framework - Node.js
│ ├── agents/ # Agent definitions
│ ├── bin/ # Installation scripts
│ ├── commands/ # Slash command definitions
│ ├── docs/ # Documentation
│ ├── get-shit-done/ # Core GSD implementation
│ │ ├── bin/ # GSD tools
│ │ ├── templates/ # Document templates
│ │ ├── workflows/ # Workflow definitions
│ │ └── references/ # Reference documentation
│ ├── hooks/ # Git hooks
│ └── scripts/ # Build/test scripts
├── gsd-openclaw/ # GSD-OpenClaw bridge - Python
│ ├── src/gsd_openclaw/ # Source package
│ │ └── prompts/ # Prompt templates
│ ├── tests/ # Test files
│ ├── skill/ # Skill definition
│ └── .planning/ # Phase planning docs
├── kb-bilibili-content/ # Bilibili content download skill
├── kb-bilibili-metadata/ # Bilibili metadata extraction skill
├── kb-deep-analyzer/ # AI deep analysis skill
├── kb-funasr/ # FunASR speech recognition skill
├── kb-link-parser/ # Multi-platform URL parser skill
├── kb-ocr-processor/ # OCR text extraction skill
├── kb-whisper/ # Whisper speech recognition skill
├── kb-whisper-diarization/ # Whisper + speaker diarization skill
├── kb-workflow/ # Knowledge base workflow orchestrator
├── kb-zhihu-content/ # Zhihu content download skill
├── kb-zhihu-metadata/ # Zhihu metadata extraction skill
├── todo-manager/ # Todo management skill
└── README.md # Project overview
```
## Directory Purposes
**kb-* Skill Directories:**
- Purpose: Independent skill modules for content processing
- Contains: Python implementation, SKILL.md specification, optional .env.example
- Key files: `*.py` (implementation), `SKILL.md` (specification)
**gsd/ Directory:**
- Purpose: Node.js-based GSD framework for project management
- Contains: Commands, agents, templates, workflows
- Key files: `package.json`, `commands/gsd/*.md`, `get-shit-done/bin/gsd-tools.cjs`
**gsd-openclaw/ Directory:**
- Purpose: Python bridge between GSD and OpenClaw chat interface
- Contains: Session management, state persistence, response formatting
- Key files: `src/gsd_openclaw/cli.py`, `src/gsd_openclaw/tmux_session.py`
**gsd-openclaw/src/gsd_openclaw/ Directory:**
- Purpose: Core Python package for GSD-OpenClaw
- Contains: CLI, session management, workspace management, state persistence
- Key files: `cli.py`, `tmux_session.py`, `workspace.py`, `state_manager.py`
**gsd-openclaw/skill/ Directory:**
- Purpose: Skill definition for GSD-OpenClaw
- Contains: SKILL.md that defines how the skill triggers and works
## Key File Locations
**Entry Points:**
- `kb-workflow/kb_workflow.py`: Knowledge base workflow orchestrator CLI
- `kb-deep-analyzer/deep_analyzer.py`: AI deep analysis CLI
- `kb-link-parser/link-parser.py`: URL parsing CLI
- `gsd-openclaw/src/gsd_openclaw/cli.py`: GSD-OpenClaw session management CLI
**Configuration:**
- `~/.kb_env`: Environment variables for knowledge base (loaded by kb-workflow)
- `gsd-openclaw/pyproject.toml`: Python package configuration
- `gsd/package.json`: Node.js package configuration
**Core Logic:**
- `gsd-openclaw/src/gsd_openclaw/tmux_session.py`: Tmux session lifecycle
- `gsd-openclaw/src/gsd_openclaw/state_manager.py`: State persistence with backup
- `gsd-openclaw/src/gsd_openclaw/workspace.py`: Project workspace management
- `kb-deep-analyzer/ai_client.py`: Multi-provider AI client
**Testing:**
- `gsd-openclaw/tests/test_*.py`: pytest test files
## Naming Conventions
**Skill Directories:**
- Pattern: `kb-{platform}-{function}` or `kb-{function}`
- Examples: `kb-bilibili-metadata`, `kb-deep-analyzer`, `kb-whisper`
**Python Files:**
- Pattern: `{platform}_{function}.py` or `{function}.py`
- Examples: `bilibili_metadata.py`, `deep_analyzer.py`, `link_parser.py`
- Note: Some use hyphens in filename (e.g., `link-parser.py`)
**Output Files:**
- Pattern: `{platform}_{id}.{type}.json`
- Examples: `bilibili_BVxxx.meta.json`, `bilibili_BVxxx.link.json`
**Skill Specifications:**
- Pattern: `SKILL.md` in each skill directory
- Format: YAML frontmatter + Markdown body
## Where to Add New Code
**New Content Platform Skill:**
- Create directory: `kb-{platform}-{function}/`
- Add implementation: `kb-{platform}-{function}/{platform}_{function}.py`
- Add specification: `kb-{platform}-{function}/SKILL.md`
- Update `kb-link-parser/link-parser.py` to recognize platform URLs
- Update `kb-workflow/SKILL.md` to include new skill in workflow
**New Processing Tool Skill:**
- Create directory: `kb-{function}/`
- Add implementation: `kb-{function}/{function}.py`
- Add specification: `kb-{function}/SKILL.md`
- Integrate with `kb-workflow/SKILL.md` if part of main pipeline
**New GSD Command:**
- Create command definition: `gsd/commands/gsd/{command-name}.md`
- Add agent if needed: `gsd/agents/gsd-{agent-name}.md`
- Update CLI in `gsd-openclaw/src/gsd_openclaw/cli.py` if needed
**New Session Management Feature:**
- Primary code: `gsd-openclaw/src/gsd_openclaw/tmux_session.py`
- Tests: `gsd-openclaw/tests/test_tmux_session.py`
- State handling: `gsd-openclaw/src/gsd_openclaw/state_manager.py`
**New AI Provider:**
- Add method to `kb-deep-analyzer/ai_client.py` `SimpleAIClient` class
- Pattern: `_call_{provider}(self, prompt, max_tokens, api_key, model, base_url)`
- Update `kb-deep-analyzer/deep_analyzer.py` `generate()` method
## Special Directories
**gsd-openclaw/.venv:**
- Purpose: Python virtual environment
- Generated: Yes (uv/pip)
- Committed: No
**gsd-openclaw/.pytest_cache:**
- Purpose: pytest cache
- Generated: Yes
- Committed: No
**gsd/.git:**
- Purpose: Separate git repository for GSD submodule
- Contains: Full GSD framework history
- Committed: As submodule reference
**__pycache__/:**
- Purpose: Python bytecode cache
- Generated: Yes
- Committed: No
**Workspace Directory (~/gsd-workspaces/):**
- Purpose: Project workspaces for GSD sessions
- Structure: `{project}/.gsd/state.json`, `{project}/code/`, `{project}/io-logs/`
- Generated: At runtime
- Committed: No (runtime data)
**Knowledge Base Directory (~/work/knowledge-base/):**
- Purpose: Working directory for content processing
- Structure: `pending/`, `metadata/`, `content/downloads/`, `analyzed/`
- Generated: At runtime via `kb_workflow.py ensure_directories()`
- Committed: No (runtime data)
**Archive Directory (~/work/zq/30.资源/AI分析/):**
- Purpose: Final archive location for analysis reports
- Structure: `YYYY-MM-DD/{platform}_{id}.md`
- Generated: At runtime
- Committed: Yes (via Git in archive workflow)
---
*Structure analysis: 2026-03-13*