Commit Graph

23 Commits

Author SHA1 Message Date
starmen1265 aa96bc963c feat(08-01): implement ZhihuCrawler with API interception
- Add ZhihuCrawler class with crawl_answer() and crawl_article() methods
- Implement ZhihuApiInterceptor for capturing /api/v4/ responses
- Add _parse_author(), _parse_metadata(), _parse_answer_data(), _parse_article_data() helpers
- Support DOM parsing fallback for zhuanlan articles
- Export ZhihuCrawler and ZhihuApiInterceptor from __init__.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 21:10:10 +08:00
starmen1265 a102a75b28 feat(08-01): create crawler data models and test scaffolding
- Add Pydantic models: AnswerData, ArticleData, ContentMetadata, AuthorInfo, CrawlResult
- Create test fixtures in conftest.py with mock browser and sample API responses
- Set up crawler module with __init__.py exports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 21:06:33 +08:00
starmen1265 d9861aa6c3 feat(07-03): implement login CLI subcommand
- Add login_command function with QR code login, cookie import, and status check
- Add login subcommand with --import-cookie and --status options
- Update CLI help text with login examples
- Fix mock paths in tests to match actual import locations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 18:10:43 +08:00
starmen1265 919a4673e6 feat(07-03): implement cookie validation functionality
- Add CookieValidationResult Pydantic model with valid, username, user_id, reason fields
- Add validate_cookie async method to ZhihuLogin class
- Validate cookie via Zhihu /api/v4/me endpoint
- Handle 200, 401, and other status codes appropriately
- Handle network errors gracefully

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 18:00:37 +08:00
starmen1265 072d83da35 feat(07-01): implement ZhihuLogin QR code login class
TDD GREEN phase - implementation passes all 8 tests:

- ZhihuLoginResult: Pydantic model for login results
- ZhihuLogin: Main class with QR code login flow
  - LOGIN_URL: https://www.zhihu.com/signin
  - CANVAS_SELECTORS: Multiple fallback selectors for QR canvas
  - _find_qrcode_canvas(): Find canvas with fallback selectors
  - _save_qrcode_image(): Save QR code to intermediate directory
  - check_login_cookie(): Detect z_c0 cookie for login state
  - get_cookies_str(): Extract cookie string from context
  - _wait_for_login(): Poll with configurable timeout (tenacity)
  - login_by_qrcode(): Main entry point for login flow

Key design decisions:
- Uses tenacity Retrying for configurable timeout
- QR code saved to intermediate_path with timestamp
- Multiple canvas selectors for robustness
- Proper browser lifecycle management

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:54:06 +08:00
starmen1265 00245c58cc feat(07-02): implement CookieManager class
- CookieManager saves cookies to ~/.kb_env file
- Updates existing ZHIHU_COOKIE line or appends new
- Preserves other configuration lines
- Sets file permissions to 600 for security
- Loads cookies from env file
- Clears get_settings cache after saving

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:50:39 +08:00
starmen1265 2360fcc99e feat(07-02): implement parse_json_cookies function
- Parse Chrome Cookie Editor JSON format
- Filter cookies by zhihu.com domain
- Handle malformed JSON with ValueError
- Handle file not found with ValueError
- Return cookie string in "name=value; name2=value2" format

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:48:51 +08:00
starmen1265 af3536aa92 test(07-02): add failing tests for Cookie JSON parsing
- Test parse_json_cookies for Chrome Cookie Editor format
- Test domain filtering for zhihu.com
- Test empty array handling
- Test malformed JSON error handling
- Test file not found error handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:48:23 +08:00
starmen1265 9fc52ffbdd refactor: integrate kb-deep-analyzer as kb_workflow.deep_analyzer package
- Move kb-deep-analyzer/deep_analyzer.py to kb_workflow/deep_analyzer/__init__.py
- Update analyze.py to import from kb_workflow.deep_analyzer directly
- Remove sys.path manipulation for external package
- DeepAnalyzer, SimpleAIClient, generate_frontmatter now available as:
  from kb_workflow.deep_analyzer import DeepAnalyzer, SimpleAIClient, generate_frontmatter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 15:28:58 +08:00
starmen1265 14b3a3c4c1 fix(analyze): correct kb-deep-analyzer path calculation
- Add one more .parent to reach correct directory level
- Fixed path: /home/.../kb-skills/kb-deep-analyzer (was /home/.../kb-skills/kb_workflow/kb-deep-analyzer)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 15:14:22 +08:00
starmen1265 244dea9371 feat(kb_workflow): add archive_file to process_command return value
PR Test / lint (pull_request) Successful in 2s
- Extract archive_file from pipeline results and include in response
- Allows callers to get the final archive path instead of only content_dir
- Fixes report display issue where content directory showed working dir instead of archive location

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 13:53:55 +08:00
starmen1265 ee0fc74378 fix(download): return author name instead of mid in _extract_author
PR Test / lint (pull_request) Successful in 1s
- Fix download processor to return actual author name (e.g. "HOLA小测佬")
  instead of B站用户ID (mid, e.g. "406636263")
- Add 5 test cases for author extraction logic
- Resolves report display issue where UP主名称 showed numeric ID

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 13:50:02 +08:00
starmen1265 72d75e3454 feat(06-03): export Pipeline class from pipeline package
- Add Pipeline import from kb_workflow.pipeline.pipeline
- Add Pipeline to __all__ exports list
- Enables: from kb_workflow.pipeline import Pipeline

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:34:26 +08:00
starmen1265 8b07ef9dcd refactor(06-03): move sys.path manipulation to lazy import in analyze.py
- Remove top-level sys.path.insert for kb-deep-analyzer
- Add module-level flag _deep_analyzer_path_added for single execution
- Move sys.path manipulation inside _run_analysis method
- External dependency import now happens lazily at runtime

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:34:06 +08:00
starmen1265 81d67b7672 fix(06-03): remove sys.path manipulation from zhihu_metadata.py
- Remove unnecessary sys.path.insert for current directory
- Use relative import for zhihu_extractor
- Package imports work correctly without path manipulation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:33:23 +08:00
starmen1265 2b305b0eb9 fix(06-03): remove sys.path manipulation from zhihu_extractor.py
- Remove unnecessary sys.path.insert for current directory
- Use pure relative import for markdown_converter
- Package imports work correctly without path manipulation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:33:02 +08:00
starmen1265 ab8655de3a fix(06-03): remove sys.path manipulation from kb_workflow.py
- Remove unnecessary sys.path.insert for project root
- Update docstring to reference kb_workflow.pipeline
- Package imports work correctly without path manipulation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:32:40 +08:00
starmen1265 c9ca1b538d refactor(06-02): update CLI and test imports to kb_workflow.pipeline
- Remove sys.path manipulation from kb_workflow.py
- Update CLI imports to use kb_workflow.pipeline prefix
- Update all test file imports and patch paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:17:10 +08:00
starmen1265 fdca690bd6 feat(06-02): move processors and platforms to kb_workflow.pipeline
- Move all processor implementations (parse, download, asr, analyze, archive)
- Move all platform adapters (base, bilibili, zhihu)
- Update all internal imports to use kb_workflow.pipeline prefix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:17:01 +08:00
starmen1265 279ae684ea feat(06-02): move pipeline core files to kb_workflow.pipeline
- Move pipeline.py, schemas.py, config.py, state.py, errors.py
- Move retry.py, idempotency.py, link_parser.py
- Create pipeline/__init__.py with updated imports
- Update all internal imports to use kb_workflow.pipeline prefix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:16:56 +08:00
starmen1265 766283366e refactor(06-01): delete original metadata directories and update tests
- Remove kb-bilibili-metadata/ and kb-zhihu-metadata/ directories
- Create kb_workflow/__init__.py to export main and process_command
- Update test_cli/test_process_command.py imports for new package structure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 00:06:01 +08:00
starmen1265 a4a7454cd0 feat(06-01): move zhihu modules to metadata directory
- Move zhihu_extractor.py, zhihu_metadata.py, zhihu_article_crawler.py, markdown_converter.py
- Update metadata/__init__.py to export ZhihuPlaywrightExtractor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:59:55 +08:00
starmen1265 e2edce2e29 feat(06-01): create metadata directory and move bilibili modules
- Rename kb-workflow to kb_workflow for Python package compatibility
- Move bilibili_extractor.py and bilibili_metadata.py to kb_workflow/metadata/
- Create metadata/__init__.py with BilibiliExporter export

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 23:59:23 +08:00