fix/crawler-proxy-config #42
@@ -22,9 +22,9 @@
|
||||
|
||||
### 内容完整性保护 (COMP)
|
||||
|
||||
- [ ] **COMP-01**: 分段不会在句子中间截断内容
|
||||
- [ ] **COMP-02**: 分段不会破坏论点的完整性（如前提-结论结构）
|
||||
- [ ] **COMP-03**: 并列结构（首先、其次、最后）不会被错误拆分
|
||||
- [x] **COMP-01**: 分段不会在句子中间截断内容
|
||||
- [x] **COMP-02**: 分段不会破坏论点的完整性（如前提-结论结构）
|
||||
- [x] **COMP-03**: 并列结构（首先、其次、最后）不会被错误拆分
|
||||
|
||||
### 主题标题生成 (TITLE)
|
||||
|
||||
@@ -103,9 +103,9 @@
|
||||
| BOUND-02 | Phase 13 | Pending |
|
||||
| BOUND-03 | Phase 13 | Pending |
|
||||
| BOUND-04 | Phase 13 | Pending |
|
||||
| COMP-01 | Phase 13 | Pending |
|
||||
| COMP-02 | Phase 13 | Pending |
|
||||
| COMP-03 | Phase 13 | Pending |
|
||||
| COMP-01 | Phase 13 | Complete |
|
||||
| COMP-02 | Phase 13 | Complete |
|
||||
| COMP-03 | Phase 13 | Complete |
|
||||
| TITLE-01 | Phase 13 | Complete |
|
||||
| TITLE-02 | Phase 13 | Complete |
|
||||
| TITLE-03 | Phase 13 | Complete |
|
||||
|
||||
@@ -219,7 +219,7 @@ Phases execute in numeric order: 1 → 2 → ... → 12 → 13 → 14 → 15
|
||||
| 10. Proxy Pool | v1.3 | 2/2 | Complete | 2026-03-18 |
|
||||
| 11. Stealth Fingerprinting | v1.4 | 2/2 | Complete | 2026-03-18 |
|
||||
| 12. SRT 简化处理器 | v1.5 | 2/2 | Complete | 2026-03-21 |
|
||||
| 13. 核心 LLM 分段功能 | 2/7 | In Progress| | - |
|
||||
| 13. 核心 LLM 分段功能 | 3/7 | In Progress| | - |
|
||||
| 14. 鲁棒性与成本控制 | v1.6 | 0/0 | Not started | - |
|
||||
| 15. 输出优化与兼容性 | v1.6 | 0/0 | Not started | - |
|
||||
|
||||
|
||||
+8
-5
@@ -3,14 +3,14 @@ gsd_state_version: 1.0
|
||||
milestone: v1.4
|
||||
milestone_name: Zhihu Anti-Crawler
|
||||
status: executing
|
||||
stopped_at: Completed 13-05 中文句边界检测
|
||||
last_updated: "2026-03-21T16:26:15.430Z"
|
||||
stopped_at: Completed 13-03-PLAN.md
|
||||
last_updated: "2026-03-21T16:26:42.744Z"
|
||||
last_activity: 2026-03-22 — Phase 13 plan created (7 plans)
|
||||
progress:
|
||||
total_phases: 15
|
||||
completed_phases: 10
|
||||
total_plans: 38
|
||||
completed_plans: 32
|
||||
completed_plans: 33
|
||||
percent: 0
|
||||
---
|
||||
|
||||
@@ -71,6 +71,8 @@ v1.6 Phase 13 decisions (from 13-CONTEXT.md):
|
||||
- **现有 SemanticSegmenter**: 完全替代 — 新代码完全替代旧代码
|
||||
- **SRTProcessor 集成**: 配置开关控制 — `enable_semantic_segmentation` 参数
|
||||
- **Pydantic Schema 位置**: `semantic_engine.py` 内
|
||||
- [Phase 13]: 确认 semantic_engine.py 已包含内容完整性保护指令（在之前的计划中实现）
|
||||
- [Phase 13]: 遵守用户决策：不添加并列结构特殊处理指令
|
||||
|
||||
### Technical Context
|
||||
|
||||
@@ -114,11 +116,12 @@ None.
|
||||
|-------|------|----------|-------|-------|------|
|
||||
| 13 | 04 | 140s (2min) | 1 | 2 | 2026-03-22 |
|
||||
| Phase 13 P05 | 2 | 1 tasks | 2 files |
|
||||
| Phase 13 P03 | 5min | 1 tasks | 1 files |
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-03-21T16:26:15.429Z
|
||||
Stopped at: Completed 13-05 中文句边界检测
|
||||
Last session: 2026-03-21T16:26:42.742Z
|
||||
Stopped at: Completed 13-03-PLAN.md
|
||||
Resume file: None
|
||||
|
||||
**Next step:** Continue with Phase 13 remaining plans (01-03, 05-07)
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
---
|
||||
phase: 13-llm
|
||||
plan: 03
|
||||
subsystem: testing
|
||||
tags: [tdd, content-completeness, sentence-boundary, argument-completeness, parallel-structure, prompt-engineering]
|
||||
|
||||
# Dependency graph
|
||||
requires:
|
||||
- phase: 13-llm
|
||||
plan: 01
|
||||
provides: [SemanticSegmentationEngine class, VideoSegment model, SegmentationResult model]
|
||||
provides:
|
||||
- 内容完整性保护测试覆盖（COMP-01, COMP-02, COMP-03）
|
||||
- 验证 SEGMENTATION_PROMPT 包含句边界和论点完整性指令
|
||||
- 确认不包含并列结构特殊处理（符合用户决策）
|
||||
affects: [13-llm-06, 13-llm-07]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns: [TDD testing pattern, mock AI client for LLM testing, regex validation for sentence boundaries]
|
||||
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- tests/test_semantic_engine.py
|
||||
|
||||
key-decisions:
|
||||
- "确认 semantic_engine.py 已包含内容完整性保护指令（在之前的计划中实现）"
|
||||
- "遵守用户决策：不添加并列结构特殊处理指令"
|
||||
|
||||
patterns-established:
|
||||
- "TDD 模式：先写测试验证行为，再实现功能"
|
||||
- "Mock AI Client 模式：使用 Mock 对象模拟 LLM 响应，避免真实 API 调用"
|
||||
- "Prompt 验证模式：通过检查 Prompt 内容确保指令完整性"
|
||||
|
||||
requirements-completed: [COMP-01, COMP-02, COMP-03]
|
||||
|
||||
# Metrics
|
||||
duration: 5min
|
||||
completed: 2026-03-22
|
||||
---
|
||||
|
||||
# Phase 13: Plan 03 - 内容完整性保护 Summary
|
||||
|
||||
**通过 Prompt 约束确保分段不破坏句子和论点完整性，使用纯 Prompt 方式不进行代码后处理**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 5 min
|
||||
- **Started:** 2026-03-22T00:23:04Z
|
||||
- **Completed:** 2026-03-22T00:28:00Z
|
||||
- **Tasks:** 1
|
||||
- **Files modified:** 1
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- 添加内容完整性保护测试（3 个测试类，6 个测试方法）
|
||||
- 验证 SEGMENTATION_PROMPT 包含所有必需的完整性保护指令
|
||||
- 确认遵守用户决策：不包含并列结构特殊处理
|
||||
- 所有 29 个测试通过（包括新增的 6 个测试）
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: 添加内容完整性保护测试** - `adc1aa3` (test)
|
||||
|
||||
**Plan metadata:** (will be added in final commit)
|
||||
|
||||
_Note: TDD tasks may have multiple commits (test → feat → refactor)_
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `tests/test_semantic_engine.py` - 添加内容完整性保护测试
|
||||
- TestSentenceBoundaryProtection (COMP-01): 句边界保护测试
|
||||
- TestArgumentCompleteness (COMP-02): 论点完整性保护测试
|
||||
- TestParallelStructurePreservation (COMP-03): 并列结构保护测试
|
||||
|
||||
## Decisions Made
|
||||
|
||||
None - 功能已在之前的计划中实现，本次仅添加测试验证
|
||||
|
||||
**确认的实现细节：**
|
||||
- 句边界保护：列举所有中文句末标点（。？！……）
|
||||
- 论点完整性：明确要求不在前提-结论结构中间截断
|
||||
- 并列结构：不特别指定，让 LLM 自然判断（符合用户决策）
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
- 内容完整性保护功能已实现并测试通过
|
||||
- 语义分段引擎的 Prompt 指令完整
|
||||
- 准备进入 Plan 06（可扩展策略架构）和 Plan 07（SRTProcessor 集成）
|
||||
|
||||
---
|
||||
*Phase: 13-llm*
|
||||
*Plan: 03*
|
||||
*Completed: 2026-03-22*
|
||||
Reference in New Issue
Block a user