Archived

This repository has been archived on 2026-08-07. You can view files and clone it. You cannot open issues or pull requests or push a commit.

Files

T

starmen1265 2561de6ad7

PR Test / lint (pull_request) Successful in 2s

Details

docs: update research for v1.3 anti-crawler optimization

- Add architecture research for proxy pool, behavior camouflage,
  captcha detection, and error recovery
- Update features, pitfalls, and stack documentation
- Add phase 10 validation checklist

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-18 20:45:14 +08:00

3.0 KiB

Raw Blame History

phase, slug, status, nyquist_compliant, wave_0_complete, created

phase	slug	status	nyquist_compliant	wave_0_complete	created
10	proxy-pool	draft	false	false	2026-03-18

Phase 10 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

Test Infrastructure

Property	Value
Framework	pytest 8.3+
Config file	pytest.ini (pythonpath = ., testpaths = tests)
Quick run command	`pytest tests/test_crawler/test_proxy_pool.py -x`
Full suite command	`pytest tests/test_crawler/ -v`
Estimated runtime	~5 seconds

Sampling Rate

After every task commit: Run pytest tests/test_crawler/test_proxy_pool.py -x
After every plan wave: Run pytest tests/test_crawler/ -v
Before /gsd:verify-work: Full suite must be green
Max feedback latency: 5 seconds

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Test Type	Automated Command	File Exists	Status
10-01-01	01	1	PROXY-01	unit	`pytest tests/test_crawler/test_proxy_pool.py::test_load_from_yaml -x`	❌ W0	⬜ pending
10-01-02	01	1	PROXY-01	unit	`pytest tests/test_crawler/test_proxy_pool.py::test_load_from_env -x`	❌ W0	⬜ pending
10-01-03	01	1	PROXY-02	unit	`pytest tests/test_crawler/test_proxy_pool.py::test_round_robin -x`	❌ W0	⬜ pending
10-02-01	02	1	PROXY-03	unit	`pytest tests/test_crawler/test_proxy_pool.py::test_fail_threshold -x`	❌ W0	⬜ pending
10-02-02	02	1	PROXY-03	unit	`pytest tests/test_crawler/test_proxy_pool.py::test_cooldown_recovery -x`	❌ W0	⬜ pending
10-02-03	02	1	PROXY-04	unit	`pytest tests/test_crawler/test_proxy_pool.py::test_all_unavailable_fallback -x`	❌ W0	⬜ pending
10-02-04	02	1	PROXY-04	integration	`pytest tests/test_crawler/test_proxy_pool.py::test_proxy_switch -x`	❌ W0	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

Wave 0 Requirements

tests/test_crawler/test_proxy_pool.py — stubs for PROXY-01~04
tests/test_crawler/conftest.py — add proxy_pool fixture (extend existing)
No framework install needed — pytest already configured

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Real proxy connection	PROXY-01~04	Requires external proxy service	Configure ~/.kb_proxies.yaml with real proxies, run crawler

All phase behaviors have automated verification for core logic. Manual verification only for integration with external proxy services.

Validation Sign-Off

All tasks have <automated> verify or Wave 0 dependencies
Sampling continuity: no 3 consecutive tasks without automated verify
Wave 0 covers all MISSING references
No watch-mode flags
Feedback latency < 5s
nyquist_compliant: true set in frontmatter

Approval: pending

3.0 KiB Raw Blame History