starmen1265/kb-skills
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
446 Commits 3 Branches 4 Tags
f8275a98462c70da9040ce2e1795bf82ec274909
Commit Graph

446 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
starmen1265 f8275a9846 docs(14): move plan 04 to wave 2 2026-03-23 22:33:53 +08:00
starmen1265 1f51ea1408 docs(14): update ROADMAP with plan list and wave structure 2026-03-23 20:57:00 +08:00
starmen1265 cefb79d311 docs(14): create phase 14 plans (5 plans + wave 0 tests) 2026-03-23 20:56:54 +08:00
starmen1265 3c9859c30d docs(14): add validation strategy 2026-03-23 20:54:44 +08:00
starmen1265 6d53ce47d4 docs(14): research phase 14 robustness and cost optimization 2026-03-23 20:53:52 +08:00
starmen1265 897ef95a87 docs(state): record phase 14 context session 2026-03-23 20:51:32 +08:00
starmen1265 2dde3a3dc8 docs(14): capture phase context 2026-03-23 20:51:29 +08:00
starmen1265 e445986d6d docs(phase-13): complete phase execution 2026-03-22 00:34:15 +08:00
starmen1265 548bb35cef docs(13-07): complete SRTProcessor integration plan 
- Create SUMMARY.md documenting SRTProcessor integration with LLM semantic segmentation
- Update STATE.md with completed plan position and performance metrics
- Update ROADMAP.md marking Phase 13 plans 01-07 as complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:33:01 +08:00
starmen1265 ef4a660faa feat(13-07): integrate SemanticSegmentationEngine into SRTProcessor 
- 添加 enable_semantic_segmentation 参数到 SRTProcessor.__init__
- 启用时使用 SemanticSegmentationEngine，禁用时使用 SemanticSegmenter
- AI 不可用时自动降级到规则分段
- 更新 _generate_markdown() 支持带标题的 VideoSegment 输出
- 调整步骤顺序：先加载元数据，再进行语义分段
- 更新 UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment]
- 添加代理方法到 SemanticSegmentationEngine 以保持向后兼容
- 创建全面的集成测试（9 个测试用例）
- 更新现有语义引擎测试以适配新 API

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:32:14 +08:00
starmen1265 a6c0293c01 docs(13-06): complete extensible strategy architecture plan 2026-03-22 00:30:41 +08:00
starmen1265 0d8ed1f22f feat(13-06): implement extensible strategy architecture for segmentation 
- Define SegmentationStrategy Protocol interface (Plan 13-06)
- Implement UnifiedSegmentationStrategy with unified approach for all video types
- Update SemanticSegmentationEngine to use strategy pattern
- Add comprehensive tests for strategy extensibility (STRAT-01, STRAT-02, STRAT-03)

Key changes:
- SegmentationStrategy Protocol defines the strategy interface
- UnifiedSegmentationStrategy implements unified approach (per user decision)
- SemanticSegmentationEngine now uses strategy injection pattern
- Tests verify strategy protocol, unified strategy, custom strategy injection, and extensibility

The architecture supports future extension (e.g., knowledge-type specific strategies)
while Phase 13 uses the unified strategy approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:29:54 +08:00
starmen1265 35f8a78d13 docs(13-02): complete semantic boundary detection plan 
完成计划 13-02 的执行和文档化：

功能验证：
- ✅ SEGMENTATION_PROMPT 包含话题转换检测指令
- ✅ extract_json_from_llm_response() 能处理多种格式
- ✅ _parse_result() 使用 Pydantic 验证 LLM 输出
- ✅ VideoSegment.start_index/end_index 验证 (ge=0)

测试覆盖：
- ✅ TestTopicBoundaryDetection (3 tests)
- ✅ TestJSONExtraction (4 tests)
- ✅ TestPydanticValidationExtended (2 tests)
- ✅ 所有 29 个测试通过

需求完成：
- ✅ BOUND-01: 话题转换点检测
- ✅ BOUND-02: 基于语义分析（非时间/字符）
- ✅ BOUND-03: 话题切换正确创建分段
- ✅ BOUND-04: 单个话题内容完整

文档：
- 创建 13-02-SUMMARY.md
- 更新 STATE.md (进度: 29%, 2/7 plans complete)
- 更新 ROADMAP.md
- 更新 REQUIREMENTS.md (标记 BOUND-* 为完成)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:27:57 +08:00
starmen1265 20a80fcb04 docs(13-01): update state, roadmap, and requirements 
- 更新 STATE.md 记录执行指标和决策
- 添加 5 个关键决策到 Accumulated Context
- 更新 ROADMAP.md 进度（计划 13-01 完成）
- 标记 VIDEO-01, VIDEO-02, VIDEO-03 为已完成
- 记录会话信息：完成 Phase 13 Plan 01

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:27:19 +08:00
starmen1265 e9c60e0704 docs(13-01): complete plan summary 
- 创建 13-01-SUMMARY.md 记录计划执行结果
- 记录 2 个 TDD 提交（RED + GREEN）
- 记录 3 个偏差（标题验证、Prompt 扩展、中文句边界测试）
- 所有测试通过（10个）
- 准备好进入计划 13-02

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:53 +08:00
starmen1265 fc1f457a7f docs(13-03): add self-check to SUMMARY.md 2026-03-22 00:26:50 +08:00
starmen1265 9dbb1a7026 docs(13-03): complete content completeness protection plan 
- 创建 SUMMARY.md：内容完整性保护功能已完成
- 更新 STATE.md：进度 87% (33/38 plans)
- 更新 ROADMAP.md：Phase 13 进展中（3/7 summaries）
- 标记需求完成：COMP-01, COMP-02, COMP-03

完成内容：
- 添加内容完整性保护测试（6 个测试方法）
- 验证 SEGMENTATION_PROMPT 包含句边界和论点完整性指令
- 确认不包含并列结构特殊处理（符合用户决策）
- 所有 29 个测试通过

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:45 +08:00
starmen1265 9a34b7df5a docs(13-05): complete Chinese sentence boundary detection plan 
- Created SUMMARY.md for plan 13-05
- Updated STATE.md with progress and metrics
- Updated ROADMAP.md with plan progress
- Marked requirements SENT-01, SENT-02, SENT-03 as complete
- Duration: 2 minutes, 1 task completed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:23 +08:00
starmen1265 adc1aa3b1d test(13-03): add content completeness protection tests 
- 添加 TestSentenceBoundaryProtection 测试类（COMP-01）
  - test_sentence_boundary_protection: 验证分段边界在标点符号后
  - test_sentence_boundary_in_prompt: 验证 Prompt 包含句边界指令

- 添加 TestArgumentCompleteness 测试类（COMP-02）
  - test_argument_completeness: 验证论点结构完整性
  - test_argument_completeness_in_prompt: 验证 Prompt 包含论点完整性指令

- 添加 TestParallelStructurePreservation 测试类（COMP-03）
  - test_parallel_structure_preserved: 验证并列结构不被错误拆分
  - test_no_special_handling_for_parallel_in_prompt: 验证遵守用户决策（不特别指定并列结构处理）

所有测试通过（29/29），semantic_engine.py 已包含所需的内容完整性保护指令：
- 第 337 行：保持句子完整性，列举所有中文句末标点（。？！……）
- 第 340 行：保持论点完整性，不在前提-结论结构中间截断
- 不包含并列结构特殊处理（符合用户决策）

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:14 +08:00
starmen1265 759503f48a feat(13-05): 实现中文句边界检测指令 
更新 SEGMENTATION_PROMPT，增强句边界检测：
- 列举所有中文句末标点（。？！……）
- 强调「自然停顿处」和「语义停顿」
- 说明「不仅依赖标点符号」

添加测试验证句边界检测效果：
- test_chinese_sentence_boundary_punctuations
- test_prompt_emphasizes_semantic_pause
- test_prompt_not_only_punctuation
- test_sentence_boundary_in_segmentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:25:40 +08:00
starmen1265 321b097492 docs(13-04): complete plan summary and state updates 
- Create 13-04-SUMMARY.md with execution details
- Update STATE.md with position and performance metrics
- Update ROADMAP.md with plan progress (1/7 complete)
- Mark TITLE-01, TITLE-02, TITLE-03 requirements complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:25:37 +08:00
starmen1265 00319f523f feat(13-01): implement video type detection and Pydantic models 
- 创建 SemanticSegmentationEngine 类
- 实现 VideoSegment 和 SegmentationResult Pydantic 模型
  - VideoSegment: title(可选), content(必填), start_index, end_index
  - SegmentationResult: video_type, segments[], actual_segment_count
  - 添加标题长度验证器（5-15字）
- 实现 _infer_video_type() 方法
  - 从元数据推断视频类型（知识类/故事类/访谈类/新闻类/vlog）
  - 支持从 title, category, tags 推断
  - 默认类型为"知识类"
- 实现 _build_prompt() 方法
  - 包含视频类型信息
  - 包含完整的分段要求（数量控制、句边界保护、标题生成等）
  - 包含 JSON 输出格式规范
- 添加 extract_json_from_llm_response() 工具函数
- 添加 AIClientProtocol 协议定义
- 添加 deferred-items.md 记录超出范围的测试

所有计划 13-01 测试通过（10个测试）

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:25:21 +08:00
starmen1265 8c2b3e079a feat(13-04): 实现主题标题生成功能 
实现 VideoSegment.title 字段和验证，Prompt 中的标题生成指令。

功能：
- VideoSegment.title: Optional[str] 字段
- @field_validator 验证标题长度（不超过 15 字）
- SEGMENTATION_PROMPT 包含标题生成指令
- LLM 自主判断是否需要标题（主题/章节有标题，论点/观点无标题）
- 测试验证标题生成和省略

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:24:58 +08:00
starmen1265 9862da7085 test(13-01): add failing tests for video type detection 
- 创建 conftest.py 共享 fixtures（MockAIClient, sample_segments）
- 创建 test_semantic_engine.py 包含 8 个测试用例
  - test_video_type_detection_from_title: 测试从标题推断类型
  - test_video_type_detection_from_category: 测试从分类推断类型
  - test_video_type_fallback_to_default: 测试默认类型回退
  - test_video_type_in_prompt: 测试类型出现在 Prompt 中
  - test_prompt_format: 测试 Prompt 格式
  - test_video_segment_model: 测试 VideoSegment 模型
  - test_segmentation_result_model: 测试 SegmentationResult 模型
  - test_model_validation: 测试字段验证

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:23:40 +08:00
starmen1265 47cd84b30a docs(state): record phase 13 planning complete 2026-03-22 00:21:40 +08:00
starmen1265 2bcaf05d52 docs(13): create phase 13 plan (7 plans) 2026-03-22 00:21:30 +08:00
starmen1265 da4b2d563a docs(13): research phase 13 LLM semantic segmentation 2026-03-22 00:19:23 +08:00
starmen1265 eecaccf288 docs(state): record phase 13 context session 2026-03-22 00:14:08 +08:00
starmen1265 6ae74f423d docs(13): capture phase context 2026-03-22 00:14:06 +08:00
starmen1265 ea2d6e4740 docs: create milestone v1.6 roadmap (3 phases, 30 requirements) 2026-03-22 00:02:53 +08:00
starmen1265 6279ab4268 docs: define milestone v1.6 requirements (30 requirements) 2026-03-21 23:58:15 +08:00
starmen1265 440c885448 docs: complete v1.6 semantic segmentation research 2026-03-21 23:53:55 +08:00
starmen1265 cf858027d4 docs: start milestone v1.6 语义分段改进 2026-03-21 23:43:58 +08:00
starmen1265 c8dcc3e09b chore: complete v1.5 milestone - B站字幕简化处理
PR Test / lint (pull_request) Successful in 2s
Details 
- Archive roadmap to milestones/v1.5-ROADMAP.md
- Archive requirements to milestones/v1.5-REQUIREMENTS.md
- Update ROADMAP.md with collapsed v1.5 entry
- Update PROJECT.md with validated requirements
- Add semantic segmentation to backlog

Delivered: B站视频简化处理流程

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-21 23:24:59 +08:00
starmen1265 d039537f1d docs(backlog): add semantic segmentation requirement 
Add requirement for improving SRT semantic segmentation:
- Current rule-based approach cannot identify topic transitions
- Document 4 potential solutions with trade-offs
- Recommended approach: two-stage with optional LLM

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v1.5 		2026-03-21 23:20:19 +08:00
starmen1265 a2ca45523b fix(srt): add punctuation requirement to ASR correction prompt 
- Update CORRECTION_PROMPT to explicitly require adding punctuation
- Fix missing punctuation in thunderstorm and lightning sections

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-21 23:20:07 +08:00
starmen1265 0752f8b38f debug: add API response logging to investigate content vs reasoning_content
PR Test / lint (pull_request) Successful in 2s
Details
2026-03-21 22:47:15 +08:00
starmen1265 4d2645876b fix(srt): add disable_reasoning parameter for ASR correction and include title in archive filename
PR Test / lint (pull_request) Successful in 1s
Details 
- SimpleAIClient: Add disable_reasoning parameter to control AI thinking mode
- ASRCorrector: Use disable_reasoning=True to get direct correction results
- ArchiveProcessor: Extract title from frontmatter and include in filename

File naming format: {YYYYMMDD}_{content_id}_{title}_analysis.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-21 22:19:14 +08:00
starmen1265 9e007432eb docs: resolve debug deep-analyzer-wrong-output
PR Test / lint (pull_request) Successful in 3s
Details
2026-03-21 21:41:30 +08:00
starmen1265 7ac4937c48 fix(deep-analyzer, srt): correct Zhipu API thinking parameter and fix metadata handling 
Root cause:
1. thinking.type="disabled" caused ASR correction accuracy degradation
2. SRTProcessor didn't flatten nested metadata structure
3. Low-quality auto-generated summary section
4. duration field type handling issue

Changes:
- DeepAnalyzer: Remove thinking.type="disabled" to enable default thinking mode
- SRTProcessor: Add metadata flattening for nested structures
- SRTProcessor: Remove low-quality summary section generation
- SRTProcessor: Add duration type checking (string or int)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-21 21:41:29 +08:00
starmen1265 1f599f0c60 fix: correct Zhipu API thinking parameter and add non-interactive archive mode
PR Test / lint (pull_request) Successful in 2s
Details
2026-03-20 00:44:41 +08:00
starmen1265 7903690882 fix(deep-analyzer): add disable_reasoning parameter to GLM API calls
PR Test / lint (pull_request) Successful in 1s
Details
2026-03-20 00:28:33 +08:00
starmen1265 fbdfc2d68a fix(12-02): add SRTProcessor to processors/__init__.py exports
PR Test / lint (pull_request) Successful in 1s
Details
2026-03-20 00:16:53 +08:00
starmen1265 04f071b3bd docs(phase-12): complete phase execution - srt-simplifier
PR Test / lint (pull_request) Successful in 2s
Details
2026-03-20 00:07:06 +08:00
starmen1265 a04d5c1ddb docs(12-02): complete Pipeline Integration plan 
Phase 12 complete:
- B站视频使用 SRTProcessor 替代 AnalyzeProcessor
- Pipeline 平台路由逻辑实现
- 24 个测试全部通过
- DeepAnalyzer 添加弃用警告

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 23:48:30 +08:00
starmen1265 a175a3242c fix(12-02): add deprecation warning to DeepAnalyzer 
- Add logging warning in DeepAnalyzer.__init__
- Warn that Bilibili video should use SRTProcessor instead
- Keep functionality intact for Zhihu and other platforms

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 23:46:08 +08:00
starmen1265 98709476ab test(12-02): add Bilibili video pipeline end-to-end tests 
- Test Bilibili video uses SRTProcessor (not AnalyzeProcessor)
- Test Bilibili article uses AnalyzeProcessor (not SRTProcessor)
- Test Zhihu answer uses AnalyzeProcessor
- Test Markdown output format verification

All 4 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 23:45:49 +08:00
starmen1265 b3b6498a71 feat(12-02): update CLI processor registration 
- Import SRTProcessor from processors module
- Add SRTProcessor to processors list
- Update PipelineConfig.steps to include 'srt' step
- Pipeline will select SRTProcessor or AnalyzeProcessor based on platform

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 23:44:05 +08:00
starmen1265 8d41a1b324 feat(12-02): modify Pipeline for platform-specific processing flow 
- Add Bilibili video detection: platform=bilibili and content_type=video
- Skip analyze step for Bilibili videos, use srt step instead
- Skip srt step for non-Bilibili-video content
- Add SRTInput import and step input creation logic
- Update _populate_skipped_step_results for srt step
- Add output_file, paragraphs, tags, corrected to result fields

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 23:43:39 +08:00
starmen1265 b312612e6c docs(12-01): complete SRT Processing Module plan 
- Create 12-01-SUMMARY.md with implementation details
- Update STATE.md with current position (Phase 12, 50% complete)
- Update ROADMAP.md with plan progress
- Mark requirements SRT-01~04, OUT-01~02, OUT-04 as complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-19 23:41:13 +08:00