This commit is contained in:
@@ -0,0 +1,460 @@
|
||||
# Phase 1: Core Infrastructure - Research
|
||||
|
||||
**Researched:** 2026-03-11
|
||||
**Domain:** tmux session management, state persistence, Python CLI
|
||||
**Confidence:** HIGH
|
||||
|
||||
## Summary
|
||||
|
||||
This phase implements the foundational infrastructure for gsd-openclaw: tmux session lifecycle management with persistent state checkpointing. The core technical stack centers on **libtmux** (v0.53.x) as the Python wrapper for tmux control, combined with JSON state files using atomic write patterns for crash-safe persistence.
|
||||
|
||||
**Primary recommendation:** Use libtmux for all tmux operations - it provides a clean Pythonic API with Server/Session/Window/Pane objects, built-in pytest fixtures for testing, and is battle-tested via tmuxp. Do not shell out to `tmux` commands directly.
|
||||
|
||||
<user_constraints>
|
||||
## User Constraints (from CONTEXT.md)
|
||||
|
||||
### Locked Decisions
|
||||
|
||||
**Session Naming:**
|
||||
- Naming format: Project name only (single-user scenario, e.g., `gsd-project-alpha`)
|
||||
- Name collision: Auto-add timestamp suffix (e.g., `-20260311-001`)
|
||||
- Display name: Support user-defined friendly names with mapping: display name -> tmux name
|
||||
- Session limit: Per-user configurable max sessions (default value by planner)
|
||||
|
||||
**State Storage:**
|
||||
- Location: Per-project independent directory `~/gsd-workspaces/{project}/.gsd/`
|
||||
- Checkpoint frequency: Auto-save every 30 seconds
|
||||
- State content:
|
||||
- Session metadata (ID, project name, creation time)
|
||||
- tmux reference (session name, pane ID)
|
||||
- Activity statistics (last active time, command count)
|
||||
- Last output snapshot
|
||||
- Atomic write: Yes (write to temp file then rename)
|
||||
- Backup strategy: Keep 1 backup (`.gsd/state.json.bak`)
|
||||
|
||||
**Workspace Structure:**
|
||||
- Root directory: `~/gsd-workspaces/{project}/`
|
||||
- Directory layout:
|
||||
```
|
||||
~/gsd-workspaces/{project}/
|
||||
├── .gsd/
|
||||
│ ├── state.json # Session state
|
||||
│ ├── state.json.bak # State backup
|
||||
│ └── session-dump/ # Session dump
|
||||
├── code/ # Project code (optional)
|
||||
└── io-logs/
|
||||
└── {session-id}.log # I/O logs
|
||||
```
|
||||
- I/O log retention: 7 days
|
||||
- Session end handling: Delete workspace when destroying session
|
||||
|
||||
**Recovery Strategy:**
|
||||
- Crash recovery: Auto-detect state file on startup, prompt user to restore
|
||||
- Orphan session handling: Auto-clean tmux sessions without state files
|
||||
- State validation: Validate state file integrity and version compatibility before restore
|
||||
- Backup usage: Auto-try backup if main state file is corrupted
|
||||
|
||||
### Claude's Discretion
|
||||
|
||||
- Exact checkpoint trigger timing (post-command vs timer)
|
||||
- State file schema version number design
|
||||
- Log file naming format
|
||||
- Specific error message wording
|
||||
|
||||
### Deferred Ideas (OUT OF SCOPE)
|
||||
|
||||
- Multi-user prefix (current single-user scenario; can add later for multi-user isolation)
|
||||
- Session pooling and LRU eviction (v2 feature)
|
||||
- Session branching and cloning (v2 feature)
|
||||
- Real-time state sync to OpenClaw control panel (Phase 5)
|
||||
|
||||
</user_constraints>
|
||||
|
||||
<phase_requirements>
|
||||
## Phase Requirements
|
||||
|
||||
| ID | Description | Research Support |
|
||||
|----|-------------|-----------------|
|
||||
| SESS-01 | User can create tmux session for a project workspace | libtmux `Server.new_session()` API |
|
||||
| SESS-02 | User can list active tmux sessions with project associations | libtmux `Server.sessions` QueryList with filter/get |
|
||||
| SESS-03 | User can destroy a tmux session when project is complete | libtmux `Session.kill()` method |
|
||||
| SESS-04 | Session state persists across server restarts via JSON checkpointing | Atomic write pattern with tempfile + os.replace |
|
||||
| SESS-05 | User can resume interrupted sessions with full context restoration | State validation + libtmux session reattachment |
|
||||
| SESS-06 | Each project has isolated workspace (separate tmux session + directory) | Workspace directory structure pattern |
|
||||
| SESS-07 | Multiple users can run projects concurrently without interference | Session naming with `gsd-` prefix (single-user for v1) |
|
||||
|
||||
</phase_requirements>
|
||||
|
||||
## Standard Stack
|
||||
|
||||
### Core
|
||||
| Library | Version | Purpose | Why Standard |
|
||||
|---------|---------|---------|--------------|
|
||||
| libtmux | 0.53.x | Python API for tmux control | Official Python wrapper, battle-tested via tmuxp, provides Server/Session/Window/Pane objects |
|
||||
| pathlib | stdlib | Path handling | Cross-platform, modern Python path API |
|
||||
| json | stdlib | State serialization | Human-readable, matches GSD patterns |
|
||||
| dataclasses | stdlib | State schema definition | Type-safe state models with minimal boilerplate |
|
||||
|
||||
### Supporting
|
||||
| Library | Version | Purpose | When to Use |
|
||||
|---------|---------|---------|-------------|
|
||||
| pytest | 8.x | Testing framework | All unit/integration tests |
|
||||
| pytest-asyncio | 0.23+ | Async test support | If checkpoint timer uses async |
|
||||
| typing | stdlib | Type hints | All function signatures |
|
||||
|
||||
### Alternatives Considered
|
||||
| Instead of | Could Use | Tradeoff |
|
||||
|------------|-----------|----------|
|
||||
| libtmux | subprocess + tmux CLI | libtmux provides typed objects, error handling, pytest fixtures; subprocess requires parsing output |
|
||||
| JSON | YAML/TOML | JSON is faster to parse, stdlib-only, matches GSD state.json pattern |
|
||||
| dataclasses | Pydantic | dataclasses is stdlib, sufficient for this use case; Pydantic adds validation overhead |
|
||||
|
||||
**Installation:**
|
||||
```bash
|
||||
pip install libtmux pytest pytest-asyncio
|
||||
```
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### Recommended Project Structure
|
||||
```
|
||||
gsd-openclaw/
|
||||
├── src/
|
||||
│ └── gsd_openclaw/
|
||||
│ ├── __init__.py
|
||||
│ ├── state_manager.py # State persistence (load/save/validate)
|
||||
│ ├── tmux_session.py # tmux session lifecycle wrapper
|
||||
│ ├── workspace.py # Workspace directory management
|
||||
│ └── cli.py # CLI entry point (create/list/destroy/resume)
|
||||
├── tests/
|
||||
│ ├── conftest.py # pytest fixtures (libtmux server)
|
||||
│ ├── test_state_manager.py # State persistence tests
|
||||
│ ├── test_tmux_session.py # Session lifecycle tests
|
||||
│ └── test_workspace.py # Workspace management tests
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
### Pattern 1: Atomic State Write
|
||||
**What:** Write state to temp file, then atomically rename to target
|
||||
**When to use:** All state.json writes to prevent corruption on crash
|
||||
**Example:**
|
||||
```python
|
||||
# Source: https://blog.gocept.com/2013/07/15/reliable-file-updates-with-python/
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
def save_state(state: dict, state_path: Path) -> None:
|
||||
"""Atomically write state to JSON file."""
|
||||
state_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write to temp file in same directory (same filesystem for atomic rename)
|
||||
fd, temp_path = tempfile.mkstemp(dir=state_path.parent, suffix='.tmp')
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
json.dump(state, f, indent=2, ensure_ascii=False)
|
||||
# Atomic replace (os.replace is atomic on POSIX)
|
||||
os.replace(temp_path, state_path)
|
||||
except Exception:
|
||||
# Clean up temp file on failure
|
||||
if os.path.exists(temp_path):
|
||||
os.unlink(temp_path)
|
||||
raise
|
||||
```
|
||||
|
||||
### Pattern 2: libtmux Session Management
|
||||
**What:** Use libtmux Server/Session objects for all tmux operations
|
||||
**When to use:** All tmux session create/list/destroy/attach operations
|
||||
**Example:**
|
||||
```python
|
||||
# Source: https://github.com/tmux-python/libtmux
|
||||
import libtmux
|
||||
|
||||
server = libtmux.Server()
|
||||
|
||||
# Create new session
|
||||
session = server.new_session(
|
||||
session_name='gsd-my-project',
|
||||
attach=False, # Don't attach (detached mode)
|
||||
window_name='main'
|
||||
)
|
||||
|
||||
# List sessions
|
||||
for s in server.sessions:
|
||||
print(f"{s.name}: {s.session_id}")
|
||||
|
||||
# Find specific session
|
||||
session = server.sessions.get(session_name='gsd-my-project')
|
||||
|
||||
# Kill session
|
||||
session.kill()
|
||||
```
|
||||
|
||||
### Pattern 3: State Schema with dataclasses
|
||||
**What:** Define typed state schema using dataclasses
|
||||
**When to use:** All state.json structure definition
|
||||
**Example:**
|
||||
```python
|
||||
from dataclasses import dataclass, asdict
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
@dataclass
|
||||
class SessionState:
|
||||
"""State schema for a gsd-openclaw session."""
|
||||
schema_version: str = "1.0"
|
||||
session_id: str = ""
|
||||
project_name: str = ""
|
||||
tmux_session_name: str = ""
|
||||
tmux_pane_id: str = ""
|
||||
created_at: str = ""
|
||||
last_active_at: str = ""
|
||||
command_count: int = 0
|
||||
last_output_snapshot: Optional[str] = None
|
||||
display_name: Optional[str] = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, data: dict) -> 'SessionState':
|
||||
return cls(**{k: v for k, v in data.items() if k in cls.__dataclass_fields__})
|
||||
```
|
||||
|
||||
### Anti-Patterns to Avoid
|
||||
- **Shelling out to tmux CLI:** Use libtmux API instead of `subprocess.run(['tmux', ...])`
|
||||
- **Non-atomic writes:** Never write directly to state.json; always use temp file + rename
|
||||
- **Hardcoded paths:** Use `~/gsd-workspaces/` via `Path.home() / "gsd-workspaces"`
|
||||
- **Session name collisions:** Always check for existing sessions before creating new ones
|
||||
|
||||
## Don't Hand-Roll
|
||||
|
||||
| Problem | Don't Build | Use Instead | Why |
|
||||
|---------|-------------|-------------|-----|
|
||||
| tmux control | subprocess + tmux commands | libtmux | Error handling, typed objects, pytest fixtures built-in |
|
||||
| Session listing | Parse `tmux list-sessions` output | `server.sessions` QueryList | Filter/get methods, type safety |
|
||||
| Atomic writes | Custom file locking | tempfile + os.replace | OS-level atomicity, proven pattern |
|
||||
| Test tmux server | Manual setup/teardown | libtmux pytest fixtures | Auto cleanup, isolated servers |
|
||||
|
||||
**Key insight:** libtmux's pytest plugin provides `server`, `session`, and `TestServer` fixtures that automatically handle setup/teardown of isolated tmux servers. This eliminates boilerplate and ensures test isolation.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Pitfall 1: Session Name Collisions
|
||||
**What goes wrong:** Creating a session with a name that already exists causes tmux error
|
||||
**Why it happens:** tmux requires unique session names per server
|
||||
**How to avoid:** Check `server.sessions.filter(session_name=...)` before creating; append timestamp suffix on collision
|
||||
**Warning signs:** `libtmux.exc.LibTmuxException` on `new_session()`
|
||||
|
||||
### Pitfall 2: State File Corruption
|
||||
**What goes wrong:** Partial writes during crash leave corrupted state.json
|
||||
**Why it happens:** Direct writes without atomic pattern can be interrupted
|
||||
**How to avoid:** Always use temp file + os.replace pattern; keep backup file
|
||||
**Warning signs:** `json.JSONDecodeError` on load; state file exists but session doesn't
|
||||
|
||||
### Pitfall 3: Orphaned tmux Sessions
|
||||
**What goes wrong:** tmux session exists but state.json is missing or corrupted
|
||||
**Why it happens:** External factors (manual tmux commands, system issues) can create sessions
|
||||
**How to avoid:** Startup scan for sessions with `gsd-` prefix without state files; offer cleanup
|
||||
**Warning signs:** `server.sessions` shows sessions not in state directory
|
||||
|
||||
### Pitfall 4: Path Expansion Issues
|
||||
**What goes wrong:** `~` in paths not expanded, or wrong home directory in tests
|
||||
**Why it happens:** String paths vs Path objects; test isolation changes HOME
|
||||
**How to avoid:** Use `Path.home()` for expansion; in tests, use pytest fixtures that set HOME
|
||||
**Warning signs:** FileNotFoundError with literal `~` in path
|
||||
|
||||
## Code Examples
|
||||
|
||||
Verified patterns from official sources:
|
||||
|
||||
### Create Session with libtmux
|
||||
```python
|
||||
# Source: https://github.com/tmux-python/libtmux
|
||||
import libtmux
|
||||
|
||||
def create_session(project_name: str) -> libtmux.Session:
|
||||
server = libtmux.Server()
|
||||
tmux_name = f"gsd-{project_name}"
|
||||
|
||||
# Check for collision and add suffix if needed
|
||||
existing = server.sessions.filter(session_name=tmux_name)
|
||||
if existing:
|
||||
from datetime import datetime
|
||||
suffix = datetime.now().strftime("-%Y%m%d-%H%M%S")
|
||||
tmux_name = f"{tmux_name}{suffix}"
|
||||
|
||||
session = server.new_session(
|
||||
session_name=tmux_name,
|
||||
attach=False,
|
||||
window_name='main'
|
||||
)
|
||||
return session
|
||||
```
|
||||
|
||||
### List Sessions with Project Info
|
||||
```python
|
||||
# Source: https://libtmux.git-pull.com/api/sessions.html
|
||||
def list_gsd_sessions() -> list[dict]:
|
||||
server = libtmux.Server()
|
||||
gsd_sessions = server.sessions.filter(session_name__startswith='gsd-')
|
||||
|
||||
return [
|
||||
{
|
||||
'tmux_name': s.name,
|
||||
'session_id': s.session_id,
|
||||
'windows': len(s.windows),
|
||||
'created': s.session_created,
|
||||
}
|
||||
for s in gsd_sessions
|
||||
]
|
||||
```
|
||||
|
||||
### Atomic State Save with Backup
|
||||
```python
|
||||
# Source: https://blog.gocept.com/2013/07/15/reliable-file-updates-with-python/
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
def save_state_with_backup(state: dict, state_path: Path) -> None:
|
||||
"""Save state atomically, maintaining backup."""
|
||||
state_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Rotate backup if main file exists
|
||||
backup_path = state_path.with_suffix('.json.bak')
|
||||
if state_path.exists():
|
||||
if backup_path.exists():
|
||||
backup_path.unlink()
|
||||
os.replace(state_path, backup_path)
|
||||
|
||||
# Atomic write new state
|
||||
fd, temp_path = tempfile.mkstemp(dir=state_path.parent, suffix='.tmp')
|
||||
try:
|
||||
with os.fdopen(fd, 'w', encoding='utf-8') as f:
|
||||
json.dump(state, f, indent=2, ensure_ascii=False)
|
||||
os.replace(temp_path, state_path)
|
||||
except Exception:
|
||||
if os.path.exists(temp_path):
|
||||
os.unlink(temp_path)
|
||||
raise
|
||||
```
|
||||
|
||||
### State Validation on Load
|
||||
```python
|
||||
def load_and_validate_state(state_path: Path) -> dict | None:
|
||||
"""Load state with validation and fallback to backup."""
|
||||
if not state_path.exists():
|
||||
# Try backup
|
||||
backup_path = state_path.with_suffix('.json.bak')
|
||||
if backup_path.exists():
|
||||
return load_and_validate_state(backup_path)
|
||||
return None
|
||||
|
||||
try:
|
||||
with open(state_path, 'r', encoding='utf-8') as f:
|
||||
state = json.load(f)
|
||||
|
||||
# Validate schema version
|
||||
if state.get('schema_version') != '1.0':
|
||||
raise ValueError(f"Unsupported schema version: {state.get('schema_version')}")
|
||||
|
||||
# Validate required fields
|
||||
required = ['session_id', 'project_name', 'tmux_session_name']
|
||||
missing = [f for f in required if not state.get(f)]
|
||||
if missing:
|
||||
raise ValueError(f"Missing required fields: {missing}")
|
||||
|
||||
return state
|
||||
|
||||
except (json.JSONDecodeError, ValueError) as e:
|
||||
# Try backup on corruption
|
||||
backup_path = state_path.with_suffix('.json.bak')
|
||||
if backup_path.exists() and backup_path != state_path:
|
||||
return load_and_validate_state(backup_path)
|
||||
raise
|
||||
```
|
||||
|
||||
## State of the Art
|
||||
|
||||
| Old Approach | Current Approach | When Changed | Impact |
|
||||
|--------------|------------------|--------------|--------|
|
||||
| subprocess tmux commands | libtmux typed API | libtmux mature since 2020 | Type safety, testability |
|
||||
| Manual pytest setup | libtmux pytest plugin | libtmux 0.30+ (2024) | Auto cleanup, isolated servers |
|
||||
| String paths | pathlib.Path | Python 3.4+ | Cross-platform, chaining |
|
||||
|
||||
**Deprecated/outdated:**
|
||||
- `session.attached_pane` / `session.attached_window`: Use `session.active_pane` / `session.active_window` (deprecated in libtmux 0.31)
|
||||
- `pane.split_window()`: Use `pane.split()` (deprecated in libtmux 0.33)
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Checkpoint Timer Implementation**
|
||||
- What we know: 30-second interval is locked in
|
||||
- What's unclear: Threading vs asyncio vs signal-based timer
|
||||
- Recommendation: Use `threading.Timer` for simplicity; asyncio if already using async elsewhere
|
||||
|
||||
2. **Schema Version Migration**
|
||||
- What we know: Need schema_version field
|
||||
- What's unclear: Migration strategy for future versions
|
||||
- Recommendation: Keep v1 simple; add migration function when v2 schema needed
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
### Test Framework
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Framework | pytest 8.x |
|
||||
| Config file | pyproject.toml (tool.pytest) |
|
||||
| Quick run command | `pytest tests/ -x -q` |
|
||||
| Full suite command | `pytest tests/ -v --cov=src/gsd_openclaw` |
|
||||
|
||||
### Phase Requirements -> Test Map
|
||||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||
|--------|----------|-----------|-------------------|-------------|
|
||||
| SESS-01 | Create tmux session | integration | `pytest tests/test_tmux_session.py::test_create_session -x` | Wave 0 |
|
||||
| SESS-02 | List active sessions | unit | `pytest tests/test_tmux_session.py::test_list_sessions -x` | Wave 0 |
|
||||
| SESS-03 | Destroy session | integration | `pytest tests/test_tmux_session.py::test_destroy_session -x` | Wave 0 |
|
||||
| SESS-04 | State persistence | unit | `pytest tests/test_state_manager.py::test_save_load_state -x` | Wave 0 |
|
||||
| SESS-05 | Session resume | integration | `pytest tests/test_tmux_session.py::test_resume_session -x` | Wave 0 |
|
||||
| SESS-06 | Workspace isolation | unit | `pytest tests/test_workspace.py::test_workspace_isolation -x` | Wave 0 |
|
||||
| SESS-07 | Concurrent user support | unit | `pytest tests/test_tmux_session.py::test_session_naming_collision -x` | Wave 0 |
|
||||
|
||||
### Sampling Rate
|
||||
- **Per task commit:** `pytest tests/ -x -q` (quick smoke test)
|
||||
- **Per wave merge:** `pytest tests/ -v` (full test suite)
|
||||
- **Phase gate:** Full suite green + coverage report before `/gsd:verify-work`
|
||||
|
||||
### Wave 0 Gaps
|
||||
- [ ] `tests/conftest.py` - libtmux pytest fixtures (server, session, tmp_path)
|
||||
- [ ] `tests/test_state_manager.py` - atomic write, backup, validation tests
|
||||
- [ ] `tests/test_tmux_session.py` - create/list/destroy/resume tests
|
||||
- [ ] `tests/test_workspace.py` - directory structure, cleanup tests
|
||||
- [ ] Framework install: `pip install pytest pytest-cov libtmux` - if none detected
|
||||
|
||||
## Sources
|
||||
|
||||
### Primary (HIGH confidence)
|
||||
- [libtmux GitHub](https://github.com/tmux-python/libtmux) - Official repository with API examples
|
||||
- [libtmux Sessions API](https://libtmux.git-pull.com/api/sessions.html) - Session class documentation
|
||||
- [libtmux Panes API](https://libtmux.git-pull.com/api/panes.html) - Pane class documentation
|
||||
- [libtmux pytest plugin](https://libtmux.git-pull.com/pytest-plugin/) - Test fixtures documentation
|
||||
- [OpenClaw SKILL.md format](https://github.com/openclaw/openclaw/blob/main/skills/skill-creator/SKILL.md) - Official skill structure
|
||||
|
||||
### Secondary (MEDIUM confidence)
|
||||
- [Atomic file updates with Python](https://blog.gocept.com/2013/07/15/reliable-file-updates-with-python/) - Write-replace pattern verified
|
||||
|
||||
### Tertiary (LOW confidence)
|
||||
- None - all critical information verified with primary sources
|
||||
|
||||
## Metadata
|
||||
|
||||
**Confidence breakdown:**
|
||||
- Standard stack: HIGH - libtmux is well-documented, actively maintained, battle-tested via tmuxp
|
||||
- Architecture: HIGH - patterns derived from official libtmux docs and established Python practices
|
||||
- Pitfalls: HIGH - based on libtmux documentation warnings and common tmux issues
|
||||
|
||||
**Research date:** 2026-03-11
|
||||
**Valid until:** 30 days - libtmux API stable, but pre-1.0 (APIs may change in 2026 per docs)
|
||||
Reference in New Issue
Block a user