starmen1265/kb-skills
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
459 Commits 3 Branches 4 Tags
bb321d9cd49c7b9f3f87eae5ccf94f9c5b991c12
Commit Graph

459 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
starmen1265 bb321d9cd4 feat(14-05): integrate cache into SRTProcessor 
- Add enable_cache parameter to SRTProcessor.__init__()
- Create SemanticCache instance when semantic segmentation is enabled
- Pass cache to UnifiedSegmentationStrategy through SemanticSegmentationEngine
- Add logging for cache status (enabled/disabled)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:40:47 +08:00
starmen1265 faff7903a4 feat(14-04): 实现长文本分块处理方法 
- 添加 _split_into_chunks() 方法，按句子边界分割长文本（5000字符阈值）
- 添加 _merge_segment_results() 方法，合并分块结果并修正索引
- 添加 _segment_single_chunk() 方法，处理单个文本块
- 添加 _segment_multiple_chunks() 方法，处理多个文本块（分块）
- 添加 _fallback_chunk() 方法，用于单个分块失败时的降级处理
- 修改 segment() 方法，添加分块判断逻辑（len(content) > 5000）

遵守用户决策：
- 分块阈值: 5000 字符
- 按句子边界分割（。！？\n）
- 直接拼接各分块结果
- start_index/end_index 相对于原文

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:40:31 +08:00
starmen1265 87a1740cd3 feat(14-05): integrate cache into UnifiedSegmentationStrategy 
- Add cache parameter to UnifiedSegmentationStrategy.__init__()
- Add cache logic to segment() method (check before LLM, save after)
- Pass cache parameter through SemanticSegmentationEngine.__init__()
- Implement missing chunking methods (_segment_single_chunk, _segment_multiple_chunks, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:40:24 +08:00
starmen1265 83b89d4725 docs(14-02): complete parse error retry and fallback plan 2026-03-23 22:39:10 +08:00
starmen1265 dfbbabceea docs(14-01): complete LLM call graceful degradation strategy 
- 添加可恢复错误检测和重试逻辑
- 实现自动降级到规则分段
- 18 个测试全部通过
- 执行时间: 219s

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:39:02 +08:00
starmen1265 5c32cfdf69 test(14-01): 实现 LLM 调用的可恢复错误检测和重试逻辑 
- 添加 _is_recoverable_error() 静态方法判断错误类型
- 可恢复错误: ConnectionError, Timeout, HTTPError (429/500/502/503/504)
- 不可恢复错误: 其他 HTTPError (4xx), ValidationError
- 重试逻辑: 仅对可恢复错误重试 1 次，立即重试（无延迟）
- 改进日志记录: 重试用 info 级别，降级用 warning 级别
- 降级时记录具体错误类型和消息

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:38:28 +08:00
starmen1265 e4c704a582 docs(14-00): complete test infrastructure plan 
- Create 14-00-SUMMARY.md documenting test infrastructure
- Update STATE.md with position and progress
- Update ROADMAP.md for Phase 14

Summary details:
- 26 tests created/modified covering ROBUST-01 to COST-03
- Test infrastructure ready for Phase 14 implementation plans
- TDD RED state established for unimplemented features
- 18 tests passing (existing functionality)
- 8 tests failing (expected - features not yet implemented)
 2026-03-23 22:38:18 +08:00
starmen1265 92c300bb21 test(14-00): add test infrastructure for robustness and cost control 
- Add comprehensive test suite in test_semantic_robustness.py
  - TestLLMFailureFallback: LLM调用失败自动降级 (ROBUST-01)
  - TestNonRecoverableErrorDetection: 不可恢复错误检测
  - TestRecoverableErrorDetection: 可恢复错误检测
  - TestRetryMechanism: 重试机制测试
  - TestFallbackToRuleBased: 降级到规则分段 (ROBUST-03)
  - TestLoggingBehavior: 日志记录行为 (ROBUST-04)
  - TestParseFailureRetry: 解析失败重试 (ROBUST-02)
  - TestExtractJSON: JSON提取功能

- Update conftest.py with Phase 14 fixtures:
  - Mock AI clients for different failure scenarios
  - Long text samples for chunking tests (COST-01)
  - Cache-related fixtures (COST-02)
  - Long subtitle segments for testing

- Add TestSemanticSegmentationToggle to test_srt_processor.py (COST-03):
  - Test enabling/disabling semantic segmentation
  - Test cache enable/disable functionality
  - Test configuration switches

All tests follow TDD RED state - failing tests for future implementation
 2026-03-23 22:37:49 +08:00
starmen1265 bf359628fc docs(14-03): complete semantic cache implementation plan 
- Created SUMMARY.md with implementation details
- Updated STATE.md with new decisions and session info
- Updated ROADMAP.md progress (1/6 plans complete)
- Marked COST-02 requirement complete

Duration: 1 min (72s)
Tasks: 1 (TDD: RED → GREEN)
Tests: 11 passed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:36:50 +08:00
starmen1265 970c13a93b test(14-02): add failing test for parse error retry and fallback 
- Test JSONDecodeError triggers retry 1 time before fallback
- Test ValidationError directly fallbacks without retry
- Test parse error logs LLM output fragment (first 500 chars)
- Test fallback logs error type and message
- Test retry uses INFO level logging

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:36:05 +08:00
starmen1265 baf7c0f521 feat(14-03): implement SemanticCache for content-based caching 
- compute_content_hash(): 计算 SHA256 内容哈希
- SemanticCache.get(): 从缓存读取 SegmentationResult
- SemanticCache.set(): 保存 SegmentationResult 到缓存
- 支持版本控制（CACHE_VERSION = '1.0'）
- 缓存存储在 .planning/cache/semantic/ 目录
- 错误容错：缓存失败不影响主流程
- 自动创建缓存目录

遵守用户决策：
- 缓存键: SHA256 内容哈希
- 缓存存储: 本地文件系统
- 缓存 TTL: 永久有效
- 缓存内容: 完整的 SegmentationResult

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:35:57 +08:00
starmen1265 b6e923c4a4 test(14-03): add failing tests for SemanticCache 
- TestComputeContentHash: 测试内容哈希计算（SHA256）
- TestSemanticCache: 测试缓存读写功能
- 测试覆盖：缓存命中/未命中、版本控制、错误处理、多缓存项

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-23 22:35:38 +08:00
starmen1265 c378a0eb32 docs(14): set nyquist compliant 2026-03-23 22:34:19 +08:00
starmen1265 f8275a9846 docs(14): move plan 04 to wave 2 2026-03-23 22:33:53 +08:00
starmen1265 1f51ea1408 docs(14): update ROADMAP with plan list and wave structure 2026-03-23 20:57:00 +08:00
starmen1265 cefb79d311 docs(14): create phase 14 plans (5 plans + wave 0 tests) 2026-03-23 20:56:54 +08:00
starmen1265 3c9859c30d docs(14): add validation strategy 2026-03-23 20:54:44 +08:00
starmen1265 6d53ce47d4 docs(14): research phase 14 robustness and cost optimization 2026-03-23 20:53:52 +08:00
starmen1265 897ef95a87 docs(state): record phase 14 context session 2026-03-23 20:51:32 +08:00
starmen1265 2dde3a3dc8 docs(14): capture phase context 2026-03-23 20:51:29 +08:00
starmen1265 e445986d6d docs(phase-13): complete phase execution 2026-03-22 00:34:15 +08:00
starmen1265 548bb35cef docs(13-07): complete SRTProcessor integration plan 
- Create SUMMARY.md documenting SRTProcessor integration with LLM semantic segmentation
- Update STATE.md with completed plan position and performance metrics
- Update ROADMAP.md marking Phase 13 plans 01-07 as complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:33:01 +08:00
starmen1265 ef4a660faa feat(13-07): integrate SemanticSegmentationEngine into SRTProcessor 
- 添加 enable_semantic_segmentation 参数到 SRTProcessor.__init__
- 启用时使用 SemanticSegmentationEngine，禁用时使用 SemanticSegmenter
- AI 不可用时自动降级到规则分段
- 更新 _generate_markdown() 支持带标题的 VideoSegment 输出
- 调整步骤顺序：先加载元数据，再进行语义分段
- 更新 UnifiedSegmentationStrategy.segment() 返回 List[VideoSegment]
- 添加代理方法到 SemanticSegmentationEngine 以保持向后兼容
- 创建全面的集成测试（9 个测试用例）
- 更新现有语义引擎测试以适配新 API

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:32:14 +08:00
starmen1265 a6c0293c01 docs(13-06): complete extensible strategy architecture plan 2026-03-22 00:30:41 +08:00
starmen1265 0d8ed1f22f feat(13-06): implement extensible strategy architecture for segmentation 
- Define SegmentationStrategy Protocol interface (Plan 13-06)
- Implement UnifiedSegmentationStrategy with unified approach for all video types
- Update SemanticSegmentationEngine to use strategy pattern
- Add comprehensive tests for strategy extensibility (STRAT-01, STRAT-02, STRAT-03)

Key changes:
- SegmentationStrategy Protocol defines the strategy interface
- UnifiedSegmentationStrategy implements unified approach (per user decision)
- SemanticSegmentationEngine now uses strategy injection pattern
- Tests verify strategy protocol, unified strategy, custom strategy injection, and extensibility

The architecture supports future extension (e.g., knowledge-type specific strategies)
while Phase 13 uses the unified strategy approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:29:54 +08:00
starmen1265 35f8a78d13 docs(13-02): complete semantic boundary detection plan 
完成计划 13-02 的执行和文档化：

功能验证：
- ✅ SEGMENTATION_PROMPT 包含话题转换检测指令
- ✅ extract_json_from_llm_response() 能处理多种格式
- ✅ _parse_result() 使用 Pydantic 验证 LLM 输出
- ✅ VideoSegment.start_index/end_index 验证 (ge=0)

测试覆盖：
- ✅ TestTopicBoundaryDetection (3 tests)
- ✅ TestJSONExtraction (4 tests)
- ✅ TestPydanticValidationExtended (2 tests)
- ✅ 所有 29 个测试通过

需求完成：
- ✅ BOUND-01: 话题转换点检测
- ✅ BOUND-02: 基于语义分析（非时间/字符）
- ✅ BOUND-03: 话题切换正确创建分段
- ✅ BOUND-04: 单个话题内容完整

文档：
- 创建 13-02-SUMMARY.md
- 更新 STATE.md (进度: 29%, 2/7 plans complete)
- 更新 ROADMAP.md
- 更新 REQUIREMENTS.md (标记 BOUND-* 为完成)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:27:57 +08:00
starmen1265 20a80fcb04 docs(13-01): update state, roadmap, and requirements 
- 更新 STATE.md 记录执行指标和决策
- 添加 5 个关键决策到 Accumulated Context
- 更新 ROADMAP.md 进度（计划 13-01 完成）
- 标记 VIDEO-01, VIDEO-02, VIDEO-03 为已完成
- 记录会话信息：完成 Phase 13 Plan 01

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:27:19 +08:00
starmen1265 e9c60e0704 docs(13-01): complete plan summary 
- 创建 13-01-SUMMARY.md 记录计划执行结果
- 记录 2 个 TDD 提交（RED + GREEN）
- 记录 3 个偏差（标题验证、Prompt 扩展、中文句边界测试）
- 所有测试通过（10个）
- 准备好进入计划 13-02

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:53 +08:00
starmen1265 fc1f457a7f docs(13-03): add self-check to SUMMARY.md 2026-03-22 00:26:50 +08:00
starmen1265 9dbb1a7026 docs(13-03): complete content completeness protection plan 
- 创建 SUMMARY.md：内容完整性保护功能已完成
- 更新 STATE.md：进度 87% (33/38 plans)
- 更新 ROADMAP.md：Phase 13 进展中（3/7 summaries）
- 标记需求完成：COMP-01, COMP-02, COMP-03

完成内容：
- 添加内容完整性保护测试（6 个测试方法）
- 验证 SEGMENTATION_PROMPT 包含句边界和论点完整性指令
- 确认不包含并列结构特殊处理（符合用户决策）
- 所有 29 个测试通过

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:45 +08:00
starmen1265 9a34b7df5a docs(13-05): complete Chinese sentence boundary detection plan 
- Created SUMMARY.md for plan 13-05
- Updated STATE.md with progress and metrics
- Updated ROADMAP.md with plan progress
- Marked requirements SENT-01, SENT-02, SENT-03 as complete
- Duration: 2 minutes, 1 task completed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:23 +08:00
starmen1265 adc1aa3b1d test(13-03): add content completeness protection tests 
- 添加 TestSentenceBoundaryProtection 测试类（COMP-01）
  - test_sentence_boundary_protection: 验证分段边界在标点符号后
  - test_sentence_boundary_in_prompt: 验证 Prompt 包含句边界指令

- 添加 TestArgumentCompleteness 测试类（COMP-02）
  - test_argument_completeness: 验证论点结构完整性
  - test_argument_completeness_in_prompt: 验证 Prompt 包含论点完整性指令

- 添加 TestParallelStructurePreservation 测试类（COMP-03）
  - test_parallel_structure_preserved: 验证并列结构不被错误拆分
  - test_no_special_handling_for_parallel_in_prompt: 验证遵守用户决策（不特别指定并列结构处理）

所有测试通过（29/29），semantic_engine.py 已包含所需的内容完整性保护指令：
- 第 337 行：保持句子完整性，列举所有中文句末标点（。？！……）
- 第 340 行：保持论点完整性，不在前提-结论结构中间截断
- 不包含并列结构特殊处理（符合用户决策）

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:26:14 +08:00
starmen1265 759503f48a feat(13-05): 实现中文句边界检测指令 
更新 SEGMENTATION_PROMPT，增强句边界检测：
- 列举所有中文句末标点（。？！……）
- 强调「自然停顿处」和「语义停顿」
- 说明「不仅依赖标点符号」

添加测试验证句边界检测效果：
- test_chinese_sentence_boundary_punctuations
- test_prompt_emphasizes_semantic_pause
- test_prompt_not_only_punctuation
- test_sentence_boundary_in_segmentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:25:40 +08:00
starmen1265 321b097492 docs(13-04): complete plan summary and state updates 
- Create 13-04-SUMMARY.md with execution details
- Update STATE.md with position and performance metrics
- Update ROADMAP.md with plan progress (1/7 complete)
- Mark TITLE-01, TITLE-02, TITLE-03 requirements complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:25:37 +08:00
starmen1265 00319f523f feat(13-01): implement video type detection and Pydantic models 
- 创建 SemanticSegmentationEngine 类
- 实现 VideoSegment 和 SegmentationResult Pydantic 模型
  - VideoSegment: title(可选), content(必填), start_index, end_index
  - SegmentationResult: video_type, segments[], actual_segment_count
  - 添加标题长度验证器（5-15字）
- 实现 _infer_video_type() 方法
  - 从元数据推断视频类型（知识类/故事类/访谈类/新闻类/vlog）
  - 支持从 title, category, tags 推断
  - 默认类型为"知识类"
- 实现 _build_prompt() 方法
  - 包含视频类型信息
  - 包含完整的分段要求（数量控制、句边界保护、标题生成等）
  - 包含 JSON 输出格式规范
- 添加 extract_json_from_llm_response() 工具函数
- 添加 AIClientProtocol 协议定义
- 添加 deferred-items.md 记录超出范围的测试

所有计划 13-01 测试通过（10个测试）

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:25:21 +08:00
starmen1265 8c2b3e079a feat(13-04): 实现主题标题生成功能 
实现 VideoSegment.title 字段和验证，Prompt 中的标题生成指令。

功能：
- VideoSegment.title: Optional[str] 字段
- @field_validator 验证标题长度（不超过 15 字）
- SEGMENTATION_PROMPT 包含标题生成指令
- LLM 自主判断是否需要标题（主题/章节有标题，论点/观点无标题）
- 测试验证标题生成和省略

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:24:58 +08:00
starmen1265 9862da7085 test(13-01): add failing tests for video type detection 
- 创建 conftest.py 共享 fixtures（MockAIClient, sample_segments）
- 创建 test_semantic_engine.py 包含 8 个测试用例
  - test_video_type_detection_from_title: 测试从标题推断类型
  - test_video_type_detection_from_category: 测试从分类推断类型
  - test_video_type_fallback_to_default: 测试默认类型回退
  - test_video_type_in_prompt: 测试类型出现在 Prompt 中
  - test_prompt_format: 测试 Prompt 格式
  - test_video_segment_model: 测试 VideoSegment 模型
  - test_segmentation_result_model: 测试 SegmentationResult 模型
  - test_model_validation: 测试字段验证

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-22 00:23:40 +08:00
starmen1265 47cd84b30a docs(state): record phase 13 planning complete 2026-03-22 00:21:40 +08:00
starmen1265 2bcaf05d52 docs(13): create phase 13 plan (7 plans) 2026-03-22 00:21:30 +08:00
starmen1265 da4b2d563a docs(13): research phase 13 LLM semantic segmentation 2026-03-22 00:19:23 +08:00
starmen1265 eecaccf288 docs(state): record phase 13 context session 2026-03-22 00:14:08 +08:00
starmen1265 6ae74f423d docs(13): capture phase context 2026-03-22 00:14:06 +08:00
starmen1265 ea2d6e4740 docs: create milestone v1.6 roadmap (3 phases, 30 requirements) 2026-03-22 00:02:53 +08:00
starmen1265 6279ab4268 docs: define milestone v1.6 requirements (30 requirements) 2026-03-21 23:58:15 +08:00
starmen1265 440c885448 docs: complete v1.6 semantic segmentation research 2026-03-21 23:53:55 +08:00
starmen1265 cf858027d4 docs: start milestone v1.6 语义分段改进 2026-03-21 23:43:58 +08:00
starmen1265 c8dcc3e09b chore: complete v1.5 milestone - B站字幕简化处理
PR Test / lint (pull_request) Successful in 2s
Details 
- Archive roadmap to milestones/v1.5-ROADMAP.md
- Archive requirements to milestones/v1.5-REQUIREMENTS.md
- Update ROADMAP.md with collapsed v1.5 entry
- Update PROJECT.md with validated requirements
- Add semantic segmentation to backlog

Delivered: B站视频简化处理流程

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-21 23:24:59 +08:00
starmen1265 d039537f1d docs(backlog): add semantic segmentation requirement 
Add requirement for improving SRT semantic segmentation:
- Current rule-based approach cannot identify topic transitions
- Document 4 potential solutions with trade-offs
- Recommended approach: two-stage with optional LLM

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v1.5 		2026-03-21 23:20:19 +08:00
starmen1265 a2ca45523b fix(srt): add punctuation requirement to ASR correction prompt 
- Update CORRECTION_PROMPT to explicitly require adding punctuation
- Fix missing punctuation in thunderstorm and lightning sections

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-21 23:20:07 +08:00
starmen1265 0752f8b38f debug: add API response logging to investigate content vs reasoning_content
PR Test / lint (pull_request) Successful in 2s
Details
2026-03-21 22:47:15 +08:00