docs(14-01): complete LLM call graceful degradation strategy
- 添加可恢复错误检测和重试逻辑 - 实现自动降级到规则分段 - 18 个测试全部通过 - 执行时间: 219s Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -46,9 +46,9 @@
|
||||
|
||||
### 鲁棒性 (ROBUST)
|
||||
|
||||
- [ ] **ROBUST-01**: LLM 调用失败时自动降级到简单规则分段
|
||||
- [x] **ROBUST-01**: LLM 调用失败时自动降级到简单规则分段
|
||||
- [ ] **ROBUST-02**: LLM 响应解析失败时自动降级到简单规则分段
|
||||
- [ ] **ROBUST-03**: AI 不可用时自动使用简单规则分段
|
||||
- [x] **ROBUST-03**: AI 不可用时自动使用简单规则分段
|
||||
- [ ] **ROBUST-04**: 降级过程有日志记录，便于问题诊断
|
||||
|
||||
### 成本控制 (COST)
|
||||
@@ -115,9 +115,9 @@
|
||||
| STRAT-01 | Phase 13 | Complete |
|
||||
| STRAT-02 | Phase 13 | Complete |
|
||||
| STRAT-03 | Phase 13 | Complete |
|
||||
| ROBUST-01 | Phase 14 | Pending |
|
||||
| ROBUST-01 | Phase 14 | Complete |
|
||||
| ROBUST-02 | Phase 14 | Pending |
|
||||
| ROBUST-03 | Phase 14 | Pending |
|
||||
| ROBUST-03 | Phase 14 | Complete |
|
||||
| ROBUST-04 | Phase 14 | Pending |
|
||||
| COST-01 | Phase 14 | Pending |
|
||||
| COST-02 | Phase 14 | Complete |
|
||||
|
||||
@@ -233,7 +233,7 @@ Phases execute in numeric order: 1 → 2 → ... → 12 → 13 → 14 → 15
|
||||
| 11. Stealth Fingerprinting | v1.4 | 2/2 | Complete | 2026-03-18 |
|
||||
| 12. SRT 简化处理器 | v1.5 | 2/2 | Complete | 2026-03-21 |
|
||||
| 13. 核心 LLM 分段功能 | 6/7 | Complete | 2026-03-21 | - |
|
||||
| 14. 鲁棒性与成本控制 | 2/6 | In Progress| | 2026-03-23 |
|
||||
| 14. 鲁棒性与成本控制 | 3/6 | In Progress| | 2026-03-23 |
|
||||
| 15. 输出优化与兼容性 | v1.6 | 0/0 | Not started | - |
|
||||
|
||||
---
|
||||
|
||||
+6
-5
@@ -3,14 +3,14 @@ gsd_state_version: 1.0
|
||||
milestone: v1.4
|
||||
milestone_name: Zhihu Anti-Crawler
|
||||
status: executing
|
||||
stopped_at: "Completed 14-00-PLAN.md: test infrastructure for robustness and cost control"
|
||||
last_updated: "2026-03-23T14:38:14.991Z"
|
||||
stopped_at: Completed 14-01-PLAN.md (LLM 调用优雅降级策略)
|
||||
last_updated: "2026-03-23T14:38:58.429Z"
|
||||
last_activity: 2026-03-22 — Phase 13 Plan 07 completed (SRTProcessor integration)
|
||||
progress:
|
||||
total_phases: 15
|
||||
completed_phases: 11
|
||||
total_plans: 44
|
||||
completed_plans: 39
|
||||
completed_plans: 40
|
||||
percent: 31
|
||||
---
|
||||
|
||||
@@ -137,11 +137,12 @@ None.
|
||||
| 13 | 07 | 4min | 1 | 4 | 2026-03-22 |
|
||||
| Phase 14-robustness-cost P03 | 72 | 1 tasks | 3 files |
|
||||
| Phase 14-robustness-cost P00 | 3min | 3 tasks | 3 files |
|
||||
| Phase 14-robustness-cost P01 | 219 | 1 tasks | 2 files |
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-03-23T14:38:14.988Z
|
||||
Stopped at: Completed 14-00-PLAN.md: test infrastructure for robustness and cost control
|
||||
Last session: 2026-03-23T14:38:58.427Z
|
||||
Stopped at: Completed 14-01-PLAN.md (LLM 调用优雅降级策略)
|
||||
Resume file: None
|
||||
|
||||
**Next step:** Phase 13 complete, continue to Phase 14 (鲁棒性) or Phase 15 (优化)
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
---
|
||||
phase: 14-robustness-cost
|
||||
plan: 01
|
||||
subsystem: LLM 调用鲁棒性
|
||||
tags: [robustness, retry, fallback, llm]
|
||||
dependency_graph:
|
||||
requires: []
|
||||
provides: [14-02]
|
||||
affects: [kb_workflow/srt/semantic_engine.py]
|
||||
tech_stack:
|
||||
added: [requests.exceptions, 重试逻辑, 降级策略]
|
||||
patterns: [TDD, 错误分类, 立即重试]
|
||||
key_files:
|
||||
created: [tests/test_semantic_robustness.py]
|
||||
modified: [kb_workflow/srt/semantic_engine.py]
|
||||
decisions: []
|
||||
metrics:
|
||||
duration: 219s
|
||||
completed_date: 2026-03-23T14:38:38Z
|
||||
---
|
||||
|
||||
# Phase 14 Plan 01: LLM 调用优雅降级策略 Summary
|
||||
|
||||
实现 LLM 调用的可恢复错误检测和重试逻辑，确保 AI 不可用时系统仍能正常运行。
|
||||
|
||||
## One-liner
|
||||
|
||||
智能重试与自动降级：可恢复错误（网络、超时、限流）重试 1 次，不可恢复错误（API 错误、验证失败）直接降级到规则分段。
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 1 - Bug] 修复 Mock 对象属性访问错误**
|
||||
- **Found during:** Task 1 (GREEN 阶段)
|
||||
- **Issue**: HTTPError 的 `response.text` 属性在测试中是 Mock 对象，不能直接切片 `[:100]`
|
||||
- **Fix**: 添加类型检查 `isinstance(response_text, str)`，只有字符串才切片
|
||||
- **Files modified**: `kb_workflow/srt/semantic_engine.py`
|
||||
- **Commit**: 5c32cfd
|
||||
|
||||
**2. [Rule 1 - Bug] 改进 HTTPError 日志消息格式**
|
||||
- **Found during:** Task 1 (GREEN 阶段)
|
||||
- **Issue**: HTTPError 的 `str()` 返回空字符串，日志缺少错误详情
|
||||
- **Fix**: 提取 `response.status_code` 和 `response.text`，格式化为 `HTTP 401 (Unauthorized)`
|
||||
- **Files modified**: `kb_workflow/srt/semantic_engine.py`
|
||||
- **Commit**: 5c32cfd
|
||||
|
||||
**3. [Rule 1 - Bug] 处理 JSON 解析错误的日志记录**
|
||||
- **Found during:** Task 1 (GREEN 阶段)
|
||||
- **Issue**: JSON 解析失败时需要记录 LLM 输出片段用于调试
|
||||
- **Fix**: 添加 `last_llm_output` 变量，记录最后 500 字符的 LLM 输出
|
||||
- **Files modified**: `kb_workflow/srt/semantic_engine.py`
|
||||
- **Commit**: 5c32cfd
|
||||
|
||||
Or: **Minor enhancements**: 改进日志记录，添加 LLM 输出片段记录，便于调试。
|
||||
|
||||
## Auth Gates
|
||||
|
||||
None.
|
||||
|
||||
## Key Changes
|
||||
|
||||
### 1. 新增 `_is_recoverable_error()` 静态方法
|
||||
|
||||
**位置**: `kb_workflow/srt/semantic_engine.py` (第 118-152 行)
|
||||
|
||||
**功能**: 判断异常是否可恢复
|
||||
|
||||
**可恢复错误**:
|
||||
- `ConnectionError` - 网络连接错误
|
||||
- `Timeout` - 请求超时
|
||||
- `HTTPError` with status 429/500/502/503/504 - 限流或服务器错误
|
||||
|
||||
**不可恢复错误**:
|
||||
- `HTTPError` with status 4xx (客户端错误) - API 错误
|
||||
- `ValidationError` - Pydantic 验证失败
|
||||
- 其他异常
|
||||
|
||||
### 2. 重构 `segment()` 方法的异常处理
|
||||
|
||||
**位置**: `kb_workflow/srt/semantic_engine.py` (第 147-256 行)
|
||||
|
||||
**重试逻辑**:
|
||||
```python
|
||||
max_attempts = 2
|
||||
for attempt in range(max_attempts):
|
||||
try:
|
||||
llm_output = self.ai_client.generate(...)
|
||||
return self._parse_result(llm_output, ...)
|
||||
except Exception as e:
|
||||
is_recoverable = self._is_recoverable_error(e)
|
||||
is_last_attempt = (attempt == max_attempts - 1)
|
||||
|
||||
if is_recoverable and not is_last_attempt:
|
||||
logger.info(f"LLM call failed (attempt {attempt + 1}/{max_attempts}): "
|
||||
f"{type(e).__name__}: {e}, retrying...")
|
||||
continue # 立即重试
|
||||
|
||||
# 不可恢复或最后一次：降级
|
||||
logger.warning(f"{error_type}: {error_msg}, 降级到规则分段")
|
||||
break
|
||||
```
|
||||
|
||||
**特点**:
|
||||
- 仅可恢复错误重试 1 次
|
||||
- 立即重试（无延迟）
|
||||
- 不可恢复错误直接降级
|
||||
- 详细的日志记录
|
||||
|
||||
### 3. 改进日志记录
|
||||
|
||||
**重试日志** (INFO 级别):
|
||||
```
|
||||
LLM call failed (attempt 1/2): ConnectionError: Connection refused, retrying...
|
||||
```
|
||||
|
||||
**降级日志** (WARNING 级别):
|
||||
- HTTP 错误: `HTTPError: HTTP 401 (Unauthorized), 降级到规则分段`
|
||||
- JSON 解析错误: `ValueError: LLM 响应中未找到 JSON 内容
|
||||
LLM 输出片段: {...}...
|
||||
降级到规则分段`
|
||||
- 其他错误: `Exception: error message, 降级到规则分段`
|
||||
|
||||
### 4. 测试覆盖
|
||||
|
||||
**新增测试文件**: `tests/test_semantic_robustness.py` (18 个核心测试)
|
||||
|
||||
**测试类**:
|
||||
1. `TestRecoverableErrorDetection` (7 tests) - 可恢复错误检测
|
||||
2. `TestNonRecoverableErrorDetection` (4 tests) - 不可恢复错误检测
|
||||
3. `TestRetryMechanism` (3 tests) - 重试机制
|
||||
4. `TestFallbackToRuleBased` (2 tests) - 降级到规则分段
|
||||
5. `TestLoggingBehavior` (2 tests) - 日志记录行为
|
||||
|
||||
**测试结果**: 18/18 passed (100%)
|
||||
|
||||
## Verification Results
|
||||
|
||||
### 核心功能验证
|
||||
|
||||
1. **可恢复错误重试**: ✅
|
||||
- ConnectionError、Timeout、429/500/502/503/504 触发重试
|
||||
- 重试次数：2 次（初始 + 1 次重试）
|
||||
- 立即重试（无延迟）
|
||||
|
||||
2. **不可恢复错误不重试**: ✅
|
||||
- 400/401/403/404 直接降级
|
||||
- 调用次数：1 次
|
||||
|
||||
3. **降级到规则分段**: ✅
|
||||
- LLM 不可用（ai_client is None）→ 规则分段
|
||||
- LLM 调用失败 → 规则分段
|
||||
- 返回 VideoSegment 格式，title 为 null
|
||||
|
||||
4. **日志记录**: ✅
|
||||
- 重试: INFO 级别，包含错误类型和消息
|
||||
- 降级: WARNING 级别，包含详细错误信息
|
||||
|
||||
### 兼容性验证
|
||||
|
||||
**现有测试**: 39/39 passed (100%)
|
||||
|
||||
- `tests/test_semantic_engine.py` - 所有语义引擎测试通过
|
||||
- 无破坏性变更
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| 执行时间 | 219 秒 (3.65 分钟) |
|
||||
| 测试通过率 | 18/18 (100%) |
|
||||
| 代码变更 | +136 行, -55 行 |
|
||||
| 新增文件 | 1 个 (test_semantic_robustness.py) |
|
||||
|
||||
## Commit Information
|
||||
|
||||
**Commit**: `5c32cfd`
|
||||
|
||||
**Message**:
|
||||
```
|
||||
test(14-01): 实现 LLM 调用的可恢复错误检测和重试逻辑
|
||||
|
||||
- 添加 _is_recoverable_error() 静态方法判断错误类型
|
||||
- 可恢复错误: ConnectionError, Timeout, HTTPError (429/500/502/503/504)
|
||||
- 不可恢复错误: 其他 HTTPError (4xx), ValidationError
|
||||
- 重试逻辑: 仅对可恢复错误重试 1 次，立即重试（无延迟）
|
||||
- 改进日志记录: 重试用 info 级别，降级用 warning 级别
|
||||
- 降级时记录具体错误类型和消息
|
||||
|
||||
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] 所有任务已执行 (1/1)
|
||||
- [x] 每个任务单独提交 (commit: 5c32cfd)
|
||||
- [x] 在计划目录中创建 SUMMARY.md
|
||||
- [x] 更新 STATE.md 的位置和决策
|
||||
- [x] 通过 `roadmap update-plan-progress` 更新 ROADMAP.md 的计划进度
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Plan 14-02: 实现分块处理逻辑，避免上下文窗口溢出
|
||||
- Plan 14-03: 实现智能缓存机制
|
||||
- Plan 14-04: 添加详细日志记录
|
||||
- Plan 14-05: 配置开关和成本控制
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
**Created files**:
|
||||
- ✅ `/home/starmen1265/work/kb-skills/tests/test_semantic_robustness.py` (646 行)
|
||||
- ✅ `/home/starmen1265/work/kb-skills/.planning/phases/14-robustness-cost/14-01-SUMMARY.md`
|
||||
|
||||
**Modified files**:
|
||||
- ✅ `/home/starmen1265/work/kb-skills/kb_workflow/srt/semantic_engine.py` (添加重试逻辑)
|
||||
|
||||
**Commits**:
|
||||
- ✅ `5c32cfd` - test(14-01): 实现 LLM 调用的可恢复错误检测和重试逻辑
|
||||
|
||||
**Tests**:
|
||||
- ✅ 18/18 核心测试通过
|
||||
- ✅ 39/39 现有测试通过
|
||||
|
||||
All checks passed.
|
||||
Reference in New Issue
Block a user