Add: windows mvp - transparent bugs not fixed
This commit is contained in:
DaZuo0122
497
docs/FRONTEND_REQUIREMENTS.md
Normal file
497
docs/FRONTEND_REQUIREMENTS.md
Normal file
@@ -0,0 +1,497 @@
|
||||
## Frontend Requirements Document — **sprimo-frontend**
|
||||
|
||||
**Document type:** Product + Engineering requirements (frontend scope only)
|
||||
**Version:** v0.1
|
||||
**Date:** 2026-02-12
|
||||
**Scope:** Bevy-based overlay renderer + local control server + minimal UI.
|
||||
**Out of scope:** Backend detection/rules engine (assumed external), cloud services.
|
||||
|
||||
---
|
||||
|
||||
# 1. Overview
|
||||
|
||||
The frontend is a **desktop overlay pet renderer** implemented in Rust (Bevy). It presents an animated character in a **transparent, borderless window** that can be **always-on-top** and optionally **click-through**. It receives control instructions from a local backend process via **localhost REST API**, applies them to its animation/state machine, and persists user preferences locally.
|
||||
|
||||
The frontend must be able to run standalone (idle animation) even if the backend is not running.
|
||||
|
||||
---
|
||||
|
||||
# 2. Goals
|
||||
|
||||
> Note: Windows and Linux should be first-hand support, macOS support is optional.
|
||||
|
||||
1. **Render a cute animated character overlay** with smooth sprite animation.
|
||||
2. Provide a **stable command interface** (REST) for backend control.
|
||||
3. Offer **essential user controls** (tray/menu + hotkeys optional) to avoid “locking” the pet in click-through mode.
|
||||
4. Persist **window position, scale, sprite pack choice, and flags**.
|
||||
5. Be **cross-platform** (Windows/macOS/Linux) with documented degradations, especially on Linux Wayland.
|
||||
|
||||
---
|
||||
|
||||
# 3. Non-Goals (Frontend v0.1)
|
||||
|
||||
* Tool activity detection (backend responsibility).
|
||||
* Online sprite gallery, accounts, telemetry.
|
||||
* Complex rigged animation (Live2D/3D). Sprite atlas only.
|
||||
* Full settings UI panel (tray + config file is enough for v0.1).
|
||||
|
||||
---
|
||||
|
||||
# 4. User Stories
|
||||
|
||||
## Core
|
||||
|
||||
* As a user, I can see the pet on my desktop immediately after launch.
|
||||
* As a user, I can drag the pet to a preferred location.
|
||||
* As a user, I can toggle click-through so the pet doesn’t block my mouse.
|
||||
* As a user, I can toggle always-on-top so the pet stays visible.
|
||||
* As a user, I can change the character (sprite pack).
|
||||
|
||||
## Backend control
|
||||
|
||||
* As a backend process, I can instruct the frontend to play specific animations and switch states via REST.
|
||||
* As a backend process, I can update position/scale/visibility.
|
||||
* As a backend process, I can query frontend health/version.
|
||||
|
||||
## Safety
|
||||
|
||||
* As a user, I can always recover control of the pet even if click-through is enabled (hotkey/tray item).
|
||||
|
||||
---
|
||||
|
||||
# 5. Functional Requirements
|
||||
|
||||
## 5.1 Window & Overlay Behavior
|
||||
|
||||
### FR-FW-1 Borderless + transparent
|
||||
|
||||
* Window must be **borderless** and **no title bar**.
|
||||
* Background must be **transparent** so only the pet is visible.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Screenshot shows only pet pixels; underlying desktop visible.
|
||||
* No OS window chrome visible.
|
||||
|
||||
### FR-FW-2 Always-on-top
|
||||
|
||||
* Support always-on-top toggle.
|
||||
* Default: ON (configurable).
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* When ON, window stays above normal windows.
|
||||
|
||||
### FR-FW-3 Click-through (mouse pass-through)
|
||||
|
||||
* Support enabling/disabling click-through:
|
||||
|
||||
* ON: mouse events pass to windows underneath.
|
||||
* OFF: pet receives mouse input (drag, context menu).
|
||||
* Must provide a **failsafe** mechanism to disable click-through without clicking the pet.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* With click-through enabled, user can click apps behind pet.
|
||||
* User can disable click-through via tray or hotkey reliably.
|
||||
|
||||
### FR-FW-4 Dragging & anchoring
|
||||
|
||||
* When click-through is OFF, user can drag the pet.
|
||||
* Dragging updates persisted position in config.
|
||||
* Optional: snapping to screen edges.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Drag works smoothly, without major jitter.
|
||||
* Relaunch restores position.
|
||||
|
||||
### FR-FW-5 Multi-monitor + DPI
|
||||
|
||||
* Window positioning must respect:
|
||||
|
||||
* multiple monitors
|
||||
* per-monitor DPI scaling
|
||||
* Position stored in a device-independent way where possible.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Pet stays at expected corner after moving between monitors.
|
||||
|
||||
### FR-FW-6 Taskbar/Dock visibility
|
||||
|
||||
* Prefer not showing a taskbar entry for the overlay window (where supported).
|
||||
* If not feasible cross-platform, document behavior.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* On Windows/macOS: overlay window ideally hidden from taskbar/dock.
|
||||
* If not implemented, doesn’t break core usage.
|
||||
|
||||
### Platform notes (requirements)
|
||||
|
||||
* **Windows:** click-through uses extended window styles (WS_EX_TRANSPARENT / layered), always-on-top via SetWindowPos.
|
||||
* **macOS:** NSWindow level + ignoresMouseEvents.
|
||||
* **Linux:** best effort:
|
||||
|
||||
* X11: possible with shape/input region.
|
||||
* Wayland: click-through may be unavailable; document limitation.
|
||||
|
||||
---
|
||||
|
||||
## 5.2 Rendering & Animation
|
||||
|
||||
### FR-ANI-1 Sprite pack format
|
||||
|
||||
Frontend must load sprite packs from local disk with:
|
||||
|
||||
* `manifest.json` (required)
|
||||
* atlas image(s) (PNG; required)
|
||||
* optional metadata (author, license, preview)
|
||||
|
||||
**Manifest must define:**
|
||||
|
||||
* sprite sheet dimensions and frame rects (or grid)
|
||||
* animations: name → ordered frame list + fps
|
||||
* anchor point (pivot) for positioning
|
||||
* optional hitbox (for drag region)
|
||||
* optional offsets per frame (advanced)
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Default bundled pack loads with no config.
|
||||
* Invalid packs fail gracefully with error message/log and fallback to default pack.
|
||||
|
||||
### FR-ANI-2 Animation playback
|
||||
|
||||
* Play loop animations (idle/dance).
|
||||
* Support one-shot animations (celebrate/error) with:
|
||||
|
||||
* duration or “until frames complete”
|
||||
* return-to-previous-state behavior
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Switching animations is immediate and consistent.
|
||||
|
||||
### FR-ANI-3 FPS control
|
||||
|
||||
* Configurable target FPS for animation updates (e.g., 30/60).
|
||||
* Rendering should remain stable.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Low CPU usage when idle animation running.
|
||||
|
||||
### FR-ANI-4 Layering
|
||||
|
||||
* Single character is mandatory for v0.1.
|
||||
* Optional: accessory layers (future).
|
||||
|
||||
---
|
||||
|
||||
## 5.3 State Machine & Command Application
|
||||
|
||||
### FR-STM-1 Core states
|
||||
|
||||
Frontend supports at minimum:
|
||||
|
||||
* `Idle`
|
||||
* `Active`
|
||||
* `Success`
|
||||
* `Error`
|
||||
* `Dragging` (internal)
|
||||
* `Hidden` (window hidden but process alive)
|
||||
|
||||
Each state maps to a default animation (configurable by sprite pack):
|
||||
|
||||
* Idle → idle animation
|
||||
* Active → typing/dance
|
||||
* Success → celebrate then return to Active/Idle
|
||||
* Error → upset then return
|
||||
|
||||
### FR-STM-2 Transition rules
|
||||
|
||||
* Backend commands can:
|
||||
|
||||
* set state directly
|
||||
* request specific animation with priority
|
||||
* Frontend must implement:
|
||||
|
||||
* debouncing/cooldown (avoid flicker)
|
||||
* priority arbitration (e.g., Error overrides Active)
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Repeated Active commands don’t restart animation constantly.
|
||||
* Error state overrides Active for N seconds.
|
||||
|
||||
### FR-STM-3 Local autonomy (no backend)
|
||||
|
||||
* If backend is absent:
|
||||
|
||||
* frontend stays in Idle with periodic idle animation
|
||||
* user controls still function
|
||||
* If backend connects later, commands apply immediately.
|
||||
|
||||
---
|
||||
|
||||
## 5.4 REST Control Server (Frontend-hosted)
|
||||
|
||||
### FR-API-1 Local binding only
|
||||
|
||||
* HTTP server must bind to `127.0.0.1` by default.
|
||||
* Port configurable; recommended default fixed port (e.g., 32145).
|
||||
|
||||
### FR-API-2 Authentication token
|
||||
|
||||
* Frontend generates a random token on first run.
|
||||
* Stored in local config directory.
|
||||
* Backend must provide `Authorization: Bearer <token>` for all non-health endpoints.
|
||||
* Health endpoint may be unauthenticated (optional).
|
||||
|
||||
### FR-API-3 Endpoints (v0.1)
|
||||
|
||||
**Required:**
|
||||
|
||||
* `GET /v1/health`
|
||||
|
||||
* returns: version, build, uptime, active_sprite_pack
|
||||
* `POST /v1/command`
|
||||
|
||||
* accept a single command
|
||||
* `POST /v1/commands`
|
||||
|
||||
* accept batch of commands
|
||||
* `GET /v1/state` (debug)
|
||||
|
||||
* current state, current animation, flags, position/scale
|
||||
|
||||
**Command types required:**
|
||||
|
||||
* `SetState { state, ttl_ms? }`
|
||||
* `PlayAnimation { name, priority, duration_ms?, interrupt? }`
|
||||
* `SetSpritePack { pack_id_or_path }`
|
||||
* `SetTransform { x?, y?, anchor?, scale?, opacity? }`
|
||||
* `SetFlags { click_through?, always_on_top?, visible? }`
|
||||
* `Toast { text, ttl_ms? }` (optional but recommended)
|
||||
|
||||
### FR-API-4 Idempotency & dedupe
|
||||
|
||||
* Commands include:
|
||||
|
||||
* `id` (ULID/UUID)
|
||||
* `ts_ms`
|
||||
* Frontend must ignore duplicate command IDs for a short window (e.g., last 5k IDs or last 10 minutes).
|
||||
|
||||
### FR-API-5 Error handling
|
||||
|
||||
* Invalid commands return 400 with error details.
|
||||
* Auth failure returns 401.
|
||||
* Server never crashes due to malformed input.
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* Fuzzing basic JSON payload does not crash.
|
||||
|
||||
---
|
||||
|
||||
## 5.5 User Controls
|
||||
|
||||
### FR-CTL-1 Tray/menu bar controls (preferred)
|
||||
|
||||
Provide tray/menu bar items:
|
||||
|
||||
* Show/Hide
|
||||
* Toggle Click-through
|
||||
* Toggle Always-on-top
|
||||
* Sprite Pack selection (at least “Default” + “Open sprite folder…”)
|
||||
* Reload sprite packs
|
||||
* Quit
|
||||
|
||||
If tray is too hard on Linux in v0.1, provide a fallback (hotkey + config).
|
||||
|
||||
### FR-CTL-2 Hotkeys (failsafe)
|
||||
|
||||
At minimum one global hotkey:
|
||||
|
||||
* Toggle click-through OR “enter interactive mode”
|
||||
|
||||
Example default:
|
||||
|
||||
* `Ctrl+Alt+P` (Windows/Linux), `Cmd+Option+P` (macOS)
|
||||
|
||||
**Acceptance**
|
||||
|
||||
* User can recover control even if pet is click-through and cannot be clicked.
|
||||
|
||||
### FR-CTL-3 Context menu (optional)
|
||||
|
||||
Right click pet (when interactive) to open a minimal menu.
|
||||
|
||||
---
|
||||
|
||||
## 5.6 Persistence & Configuration
|
||||
|
||||
### FR-CFG-1 Config storage
|
||||
|
||||
* Store config in OS-appropriate directory:
|
||||
|
||||
* Windows: `%APPDATA%/sprimo/config.toml`
|
||||
* macOS: `~/Library/Application Support/sprimo/config.toml`
|
||||
* Linux: `~/.config/sprimo/config.toml`
|
||||
|
||||
### FR-CFG-2 Config fields
|
||||
|
||||
* window:
|
||||
|
||||
* position (x,y) + monitor id (best-effort)
|
||||
* scale
|
||||
* always_on_top
|
||||
* click_through
|
||||
* visible
|
||||
* animation:
|
||||
|
||||
* fps
|
||||
* idle_timeout_ms
|
||||
* sprite:
|
||||
|
||||
* selected_pack
|
||||
* sprite_packs_dir
|
||||
* api:
|
||||
|
||||
* port
|
||||
* auth_token
|
||||
* logging:
|
||||
|
||||
* level
|
||||
|
||||
### FR-CFG-3 Live reload (nice-to-have)
|
||||
|
||||
* v0.1: reload on tray action (“Reload config”).
|
||||
* v0.2+: auto reload.
|
||||
|
||||
---
|
||||
|
||||
## 5.7 Logging & Diagnostics
|
||||
|
||||
### FR-LOG-1 Logging
|
||||
|
||||
* Log to file + console (configurable).
|
||||
* Include:
|
||||
|
||||
* API requests summary
|
||||
* command application
|
||||
* sprite pack load errors
|
||||
* platform overlay operations outcomes
|
||||
|
||||
### FR-LOG-2 Diagnostics endpoint (optional)
|
||||
|
||||
* `GET /v1/diag` returns recent errors and platform capability flags.
|
||||
|
||||
---
|
||||
|
||||
# 6. Non-Functional Requirements
|
||||
|
||||
## Performance
|
||||
|
||||
* Idle: < 2% CPU on typical dev laptop (target).
|
||||
* Memory: should not grow unbounded; texture caching bounded.
|
||||
|
||||
## Reliability
|
||||
|
||||
* Should run for 8 hours without crash under command load.
|
||||
* If REST server fails to bind port, show clear error and fallback behavior.
|
||||
|
||||
## Security
|
||||
|
||||
* Localhost only binding.
|
||||
* Token auth by default.
|
||||
* No file system access beyond sprite/config/log directories.
|
||||
|
||||
## Compatibility
|
||||
|
||||
* Windows 10/11
|
||||
* macOS 12+ recommended
|
||||
* Linux:
|
||||
|
||||
* X11 supported
|
||||
* Wayland: documented limitations, still runs
|
||||
|
||||
---
|
||||
|
||||
# 7. Platform Capability Matrix (deliverable)
|
||||
|
||||
Frontend must expose in logs (and optionally `/v1/health`) capability flags:
|
||||
|
||||
* `supports_click_through`
|
||||
* `supports_transparency`
|
||||
* `supports_tray`
|
||||
* `supports_global_hotkey`
|
||||
* `supports_skip_taskbar`
|
||||
|
||||
Example:
|
||||
|
||||
* Windows: all true
|
||||
* macOS: all true
|
||||
* Linux X11: most true
|
||||
* Linux Wayland: click-through likely false, skip-taskbar variable
|
||||
|
||||
---
|
||||
|
||||
# 8. Acceptance Test Plan (v0.1)
|
||||
|
||||
## Window
|
||||
|
||||
1. Launch: window appears borderless & transparent.
|
||||
2. Drag: with click-through OFF, drag updates position; restart restores.
|
||||
3. Click-through: toggle via hotkey; pet becomes non-interactive; toggle back works.
|
||||
4. Always-on-top: verify staying above typical apps.
|
||||
|
||||
## Animation
|
||||
|
||||
1. Default pack: idle animation loops.
|
||||
2. Command: `PlayAnimation("dance")` plays immediately.
|
||||
3. One-shot: `Success` plays then returns to previous.
|
||||
|
||||
## API
|
||||
|
||||
1. Health returns correct version/build.
|
||||
2. Auth required for commands; invalid token rejected.
|
||||
3. Batch endpoint applies multiple commands in order.
|
||||
4. Malformed JSON returns 400, app remains running.
|
||||
|
||||
## Persistence
|
||||
|
||||
1. Settings persist across restart.
|
||||
2. Missing sprite pack falls back to default.
|
||||
|
||||
---
|
||||
|
||||
# 9. Implementation Constraints / Suggested Stack (not mandatory but recommended)
|
||||
|
||||
* Renderer/engine: **Bevy 2D**
|
||||
* REST server: **axum + tokio** in background thread
|
||||
* Shared protocol: `protocol` crate with `serde` structs
|
||||
* Platform overlay: separate crate/module with per-OS implementations:
|
||||
|
||||
* Windows: `windows` crate (Win32 APIs)
|
||||
* macOS: `objc2`/`cocoa` bindings
|
||||
* Linux X11: `x11rb` (best effort)
|
||||
* Config: `toml`
|
||||
* IDs: `ulid` or `uuid`
|
||||
|
||||
---
|
||||
|
||||
# 10. Open Questions (Frontend)
|
||||
|
||||
1. Do we require Linux Wayland support beyond “runs but limited overlay features”?
|
||||
2. Do we want multiple pets (multiple windows) in v0.1 or later?
|
||||
3. Do we embed a default sprite pack (license?), or ship an empty skeleton?
|
||||
|
||||
---
|
||||
|
||||
If you want, I can turn this into:
|
||||
|
||||
* a `docs/FRONTEND_REQUIREMENTS.md`
|
||||
* plus a concrete **REST API spec** (OpenAPI-like) and a **sprite-pack manifest schema** with examples (JSON).
|
||||
Reference in New Issue
Block a user