This repository has been archived on 2026-08-07. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
kb-skills/.planning/phases/15-output-compat/15-RESEARCH.md
T

366 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Phase 15: 输出优化与兼容性 - Research
**研究时间:** 2026-03-23
**技术领域:** 配置管理 (pydantic-settings)、输出格式验证、向后兼容性
**置信度:** HIGH
## Summary
Phase 15 专注于配置开关集成和向后兼容性验证。核心任务是将 `enable_semantic_segmentation` 添加到 KBSettings,并确保配置变更不影响现有功能。研究发现 Phase 13/14 已实现所需的输出格式(`_generate_markdown()` 支持标题和纯文本两种模式),本阶段主要是配置集成和验证工作。
**主要建议:** 使用 pydantic-settings 的标准模式添加布尔配置字段,通过环境变量 `KB_ENABLE_SEMANTIC_SEGMENTATION` 控制,默认值为 `True` 以提供开箱即用的语义分段体验。
## User Constraints (from CONTEXT.md)
### Locked Decisions
- **配置开关位置**: 添加到 KBSettings — 将 `enable_semantic_segmentation` 添加到 `kb_workflow/pipeline/config.py` 中的 `KBSettings` 类
- **配置方式**: 用户可在 `~/.kb_env` 文件中设置 `KB_ENABLE_SEMANTIC_SEGMENTATION=true`
- **环境变量命名**: `KB_ENABLE_SEMANTIC_SEGMENTATION` — 使用全名,清晰明确
- **风格一致**: 与现有配置变量命名保持一致(KB_ 前缀,全大写下划线分隔)
- **默认行为**: 默认启用 — 当用户未配置时,默认值为 `True`(启用语义分段)
- **API Key 检查保留**: 即使配置启用,如果 AI API key 未配置,代码仍会自动降级到规则分段
### Claude's Discretion
- KBSettings 中字段的具体类型定义(bool Field with default)
- 是否需要添加 enable_cache 相关的配置选项到 KBSettings
- 配置变更后的日志记录级别
### Deferred Ideas (OUT OF SCOPE)
- 缓存配置选项 — 是否需要将 enable_cache 也添加到 KBSettings(当前只有构造函数参数)
- 分段粒度配置 — 未来可能允许用户配置目标分段数量(当前硬编码为 5-8)
- 环境变量简化 — 是否需要更短的别名(如 KB_LLMSeg)作为备选
## Phase Requirements
| ID | Description | Research Support |
|----|-------------|-----------------|
| OUTPUT-01 | 主题/章节分段输出带标题的 Markdown 结构 | 已在 Phase 13 实现,`_generate_markdown()` 支持 VideoSegment.title → H2 标题 |
| OUTPUT-02 | 论点/观点分段输出纯文本,保持原有风格 | 已实现,title=null 时不输出标题,仅输出 content |
| OUTPUT-03 | 输出与现有 SRTProcessor 兼容,不影响其他平台处理 | 已验证,规则分段继续使用 "## 内容" 标题,AnalyzeProcessor 独立不受影响 |
| CONFIG-01 | LLM 分段功能可通过配置启用/禁用 | 研究提供 pydantic-settings 集成方案 |
| CONFIG-02 | 向后兼容,不破坏现有简单规则分段 | 已验证,enable_semantic_segmentation=False 时使用规则分段器 |
| CONFIG-03 | 知乎/文章平台继续使用 AnalyzeProcessor,不受影响 | 已验证,AnalyzeProcessor 不依赖 SRTProcessor 的分段逻辑 |
## Standard Stack
### Core
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| pydantic-settings | 2.13.1 | 从环境变量和 .kb_env 文件加载配置 | 项目已使用,提供 BaseSettings 基类和 env_prefix 机制 |
| pydantic | 2.12.5 | 配置字段验证和类型安全 | 项目已使用,提供 Field 和 Field 定义 |
| pytest | (现有版本) | 测试框架 | 项目已有完整测试基础设施 |
### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| unittest.mock | (stdlib) | 测试时模拟环境变量 | 测试配置加载逻辑 |
| pytest fixtures | (现有) | 共享测试资源和设置 | 现有 conftest.py 模式 |
### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| pydantic-settings | 直接读取 os.getenv | 失去类型验证、自动文档、环境文件支持 |
| bool Field | str Field + 自定义解析 | 增加复杂度,pydantic 已支持布尔字符串解析 |
**Installation:**
```bash
# 无需安装,项目已包含所有依赖
# 如需验证安装:
python3 -c "import pydantic_settings; print(pydantic_settings.__version__)"
python3 -c "import pydantic; print(pydantic.__version__)"
```
## Architecture Patterns
### Recommended Project Structure
```
kb_workflow/
├── pipeline/
│ ├── config.py # 添加 enable_semantic_segmentation 字段
│ └── processors/
│ ├── srt.py # 读取配置并传递给构造函数
│ └── analyze.py # 不受影响,独立处理器
tests/
├── test_pipeline/
│ └── test_config.py # 添加新字段测试
└── test_processors/
├── test_srt.py # 添加配置集成测试
└── conftest.py # 可能需要添加配置相关的 fixtures
```
### Pattern 1: Pydantic Settings Field with Default
**What:** 使用 `Field(default=True)` 定义布尔配置字段
**When to use:** 添加可选的布尔配置项,支持环境变量覆盖
**Example:**
```python
# Source: https://docs.pydantic.dev/2.0/api/pydantic_settings/
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict
class KBSettings(BaseSettings):
model_config = SettingsConfigDict(
env_file=str(Path.home() / ".kb_env"),
env_file_encoding="utf-8",
env_prefix="KB_",
extra="ignore",
)
# 新增字段
enable_semantic_segmentation: bool = Field(
default=True,
description="启用 LLM 语义分段(默认启用)"
)
```
**Environment Variable Resolution:**
```bash
# ~/.kb_env 文件
KB_ENABLE_SEMANTIC_SEGMENTATION=true
# 或环境变量
export KB_ENABLE_SEMANTIC_SEGMENTATION=true
# Pydantic 自动解析以下字符串为布尔 True:
# - "true", "True", "TRUE", "t", "yes", "y", "1"
# - False: "false", "False", "FALSE", "f", "no", "n", "0"
```
### Pattern 2: Configuration Integration in Processor
**What:** 在 Processor 构造函数中从 `get_settings()` 读取配置
**When to use:** 需要将全局配置传递给处理器
**Example:**
```python
# kb_workflow/pipeline/processors/srt.py (修改示例)
from kb_workflow.pipeline.config import get_settings
class SRTProcessor(BaseProcessor):
def __init__(
self,
state_manager: Optional[WorkflowStateManager] = None,
enable_correction: bool = True,
enable_semantic_segmentation: Optional[bool] = None, # 允许覆盖
enable_cache: bool = True
):
super().__init__(state_manager)
# 如果未明确指定,从配置读取
if enable_semantic_segmentation is None:
settings = get_settings()
enable_semantic_segmentation = settings.enable_semantic_segmentation
self.enable_semantic_segmentation = enable_semantic_segmentation
# 记录配置状态
logger.info(
f"SRTProcessor initialized: "
f"semantic_segmentation={'enabled' if self.enable_semantic_segmentation else 'disabled'}"
)
```
### Anti-Patterns to Avoid
- **硬编码默认值在调用处:** 应从 KBSettings 统一读取,而非在 `cli.py` 或 `pipeline.py` 中硬编码
- **配置验证逻辑分散:** 应依赖 pydantic 的自动验证,而非手写验证代码
- **忽略环境变量前缀:** 所有配置项必须遵循 `KB_` 前缀约定
## Don't Hand-Roll
| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| 配置文件解析 | 手动读取 ~/.kb_env | pydantic-settings BaseSettings | 自动处理 env 文件、环境变量、类型验证 |
| 布尔字符串解析 | 手动转换 "true"/"false" | pydantic bool Field | 自动支持多种布尔表示,错误处理 |
| 配置缓存 | 单例模式手写 | @lru_cache on get_settings() | 已有实现,线程安全 |
**Key insight:** pydantic-settings 提供了生产级配置管理的所有必要功能,手写方案只会增加维护负担和出错风险。
## Common Pitfalls
### Pitfall 1: 配置未生效由于缓存
**What goes wrong:** 修改环境变量或 .kb_env 后,`get_settings()` 返回旧值
**Why it happens:** `get_settings()` 使用 `@lru_cache`,同一进程内配置只加载一次
**How to avoid:** 测试时调用 `get_settings.cache_clear()`,或重启进程
**Warning signs:** 配置变更后行为不变,日志显示旧配置值
### Pitfall 2: 布尔字段类型错误
**What goes wrong:** 环境变量设为 "enabled"/"disabled" 导致 ValidationError
**Why it happens:** Pydantic bool 只接受标准布尔值(true/false, yes/no, 1/0 等)
**How to avoid:** 使用标准布尔值,或在 Field 中添加自定义验证器
**Warning signs:** 启动时抛出 `ValidationError: true|false expected`
### Pitfall 3: 配置作用域混淆
**What goes wrong:** Processor 构造函数参数与环境变量配置冲突
**Why it happens:** 未明确参数优先级(init_kwargs > env_vars > defaults)
**How to avoid:** 文档化参数优先级,允许显式参数覆盖配置
**Warning signs:** 用户设置环境变量但行为不符预期
## Code Examples
Verified patterns from official sources:
### Pydantic Settings Bool Field
```python
# Source: https://docs.pydantic.dev/2.0/api/pydantic_settings/
from pydantic import Field
from pydantic_settings import BaseSettings
class KBSettings(BaseSettings):
# 布尔字段,默认 True,支持环境变量
enable_feature: bool = Field(
default=True,
description="功能开关描述"
)
# 使用方式
# 1. 默认值
settings = KBSettings() # enable_feature = True
# 2. 环境变量覆盖
# export KB_ENABLE_FEATURE=false
settings = KBSettings() # enable_feature = False
# 3. 构造函数参数覆盖
settings = KBSettings(enable_feature=False) # enable_feature = False
```
### Testing Configuration with Environment Variables
```python
# Source: existing tests/test_pipeline/test_config.py pattern
import os
import pytest
from unittest.mock import patch
def test_enable_semantic_segmentation_from_env():
"""Test KB_ENABLE_SEMANTIC_SEGMENTATION env var is respected"""
with patch.dict(os.environ, {
"KB_BASE_PATH": "/tmp/kb",
"KB_ARCHIVE_PATH": "/tmp/archive",
"KB_ENABLE_SEMANTIC_SEGMENTATION": "false"
}, clear=False):
get_settings.cache_clear()
settings = get_settings()
assert settings.enable_semantic_segmentation is False
def test_enable_semantic_segmentation_default():
"""Test default value is True when env var not set"""
with patch.dict(os.environ, {
"KB_BASE_PATH": "/tmp/kb",
"KB_ARCHIVE_PATH": "/tmp/archive"
}, clear=True):
get_settings.cache_clear()
settings = get_settings()
assert settings.enable_semantic_segmentation is True
```
### Verifying Backward Compatibility
```python
# Source: existing SRTProcessor code pattern
def test_srt_processor_with_semantic_disabled():
"""Test that disabling semantic segmentation uses rule-based"""
processor = SRTProcessor(
enable_semantic_segmentation=False,
enable_correction=False
)
# 验证使用规则分段器
from kb_workflow.srt.segmenter import SemanticSegmenter
assert isinstance(processor.segmenter, SemanticSegmenter)
assert not isinstance(processor.segmenter, SemanticSegmentationEngine)
def test_srt_processor_respects_config():
"""Test SRTProcessor reads from KBSettings"""
with patch.dict(os.environ, {
"KB_BASE_PATH": "/tmp/kb",
"KB_ARCHIVE_PATH": "/tmp/archive",
"KB_ENABLE_SEMANTIC_SEGMENTATION": "false"
}):
get_settings.cache_clear()
processor = SRTProcessor() # 应从配置读取
# 验证配置被正确应用
assert processor.enable_semantic_segmentation is False
```
## State of the Art
| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| 硬编码配置 | pydantic-settings + .kb_env | 项目初始 | 集中管理、类型安全 |
| 构造函数参数 | 构造函数参数 + 环境变量 | Phase 15 | 用户可通过配置文件控制,无需修改代码 |
**Deprecated/outdated:**
- 手动解析环境变量(`os.getenv()`)— 应使用 pydantic-settings 自动解析
- 配置文件自定义格式(JSON/YAML)— .kb_env dotenv 格式为标准
## Validation Architecture
> Nyquist verification enabled (workflow.nyquist_validation: true)
### Test Framework
| Property | Value |
|----------|-------|
| Framework | pytest (版本未知,项目现有) |
| Config file | pytest.ini (pythonpath = ., testpaths = tests) |
| Quick run command | `pytest tests/test_pipeline/test_config.py -x -v` |
| Full suite command | `pytest tests/ -x` |
### Phase Requirements → Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| OUTPUT-01 | 带 H2 标题的 Markdown 输出 | integration | `pytest tests/test_srt_processor.py::test_generate_markdown_with_titles -x` | ✅ Phase 13 |
| OUTPUT-02 | 纯文本输出(title=null) | integration | `pytest tests/test_srt_processor.py::test_generate_markdown_without_titles -x` | ✅ Phase 13 |
| OUTPUT-03 | 规则分段输出 "## 内容" | integration | `pytest tests/test_srt_processor.py::test_generate_markdown_rule_based -x` | ✅ Phase 13 |
| CONFIG-01 | KBSettings 新字段加载 | unit | `pytest tests/test_pipeline/test_config.py::test_kbsettings_enable_semantic_segmentation_field -x` | ❌ Wave 0 |
| CONFIG-01 | 环境变量覆盖默认值 | unit | `pytest tests/test_pipeline/test_config.py::test_enable_semantic_segmentation_from_env -x` | ❌ Wave 0 |
| CONFIG-02 | 禁用时使用规则分段 | integration | `pytest tests/test_processors/test_srt.py::test_srt_processor_with_semantic_disabled -x` | ❌ Wave 0 |
| CONFIG-03 | AnalyzeProcessor 不受影响 | integration | `pytest tests/test_processors/test_analyze.py -x` | ✅ 已存在 |
### Sampling Rate
- **Per task commit:** `pytest tests/test_pipeline/test_config.py -x -v`
- **Per wave merge:** `pytest tests/test_pipeline/test_config.py tests/test_processors/test_srt.py -x`
- **Phase gate:** Full suite green before `/gsd:verify-work`
### Wave 0 Gaps
- [ ] `tests/test_pipeline/test_config.py` — 添加新字段测试(CONFIG-01)
- [ ] `tests/test_processors/test_srt.py` — 添加配置集成测试(CONFIG-01, CONFIG-02)
- [ ] `tests/test_processors/test_srt.py` — 验证向后兼容性(CONFIG-02, OUTPUT-03)
- [ ] Framework install: 无需安装(现有 pytest 环境)
- [ ] Shared fixtures: `tests/test_processors/conftest.py` 已存在,可能需要添加配置 mock
## Open Questions
1. **enable_cache 是否也需要添加到 KBSettings?**
- What we know: 当前 `enable_cache` 只有构造函数参数,未集成到 KBSettings
- What's unclear: 用户是否需要全局控制缓存开关
- Recommendation: Phase 15 范围内不添加,保留为构造函数参数(符合 Claude's Discretion)
2. **配置变更日志级别**
- What we know: SRTProcessor 初始化时已有日志记录
- What's unclear: 应使用 INFO 还是 DEBUG 级别
- Recommendation: INFO 级别(用户可见,便于调试配置问题)
## Sources
### Primary (HIGH confidence)
- [pydantic-settings API Documentation](https://docs.pydantic.dev/2.0/api/pydantic_settings/) - BaseSettings, Field, env_prefix
- [Pydantic Settings Usage](https://docs.pydantic.dev/latest/concepts/pydantic_settings/) - 配置文件和环境变量加载
- [kb_workflow/pipeline/config.py](file:///home/starmen1265/work/kb-skills/kb_workflow/pipeline/config.py) - 现有 KBSettings 实现
- [kb_workflow/pipeline/processors/srt.py](file:///home/starmen1265/work/kb-skills/kb_workflow/pipeline/processors/srt.py) - SRTProcessor 构造函数和配置集成
- [kb_workflow/srt/semantic_engine.py](file:///home/starmen1265/work/kb-skills/kb_workflow/srt/semantic_engine.py) - _generate_markdown() 实现
### Secondary (MEDIUM confidence)
- [pytest documentation](https://docs.pytest.org/) - 测试框架标准用法
- Existing test files: `tests/test_pipeline/test_config.py`, `tests/test_srt_processor.py` - 项目测试模式
### Tertiary (LOW confidence)
- None (所有研究基于官方文档或项目代码)
## Metadata
**Confidence breakdown:**
- Standard stack: HIGH - 基于项目现有依赖和官方文档
- Architecture: HIGH - pydantic-settings 是行业标准,项目已采用
- Pitfalls: HIGH - 基于已知 pydantic 使用陷阱和项目测试经验
- Output format: HIGH - 基于代码审查,Phase 13 已实现所需功能
**Research date:** 2026-03-23
**Valid until:** 30 days (pydantic-settings 稳定版本,项目配置模式成熟)