Add: dummy backend for behavioural testing
This commit is contained in:
@@ -2,9 +2,9 @@
|
||||
|
||||
File location:
|
||||
|
||||
- Windows: `%APPDATA%/sprimo/config.toml`
|
||||
- macOS: `~/Library/Application Support/sprimo/config.toml`
|
||||
- Linux: `~/.config/sprimo/config.toml`
|
||||
- Windows: `%APPDATA%/sprimo/config/config.toml`
|
||||
- macOS: `~/Library/Application Support/sprimo/config/config.toml`
|
||||
- Linux: `~/.config/sprimo/config/config.toml`
|
||||
|
||||
## Schema
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ Date: 2026-02-12
|
||||
| Global failsafe | Implemented (Windows) | Global recovery hotkey `Ctrl+Alt+P` forces visibility and top-most recovery |
|
||||
| Embedded default pack | Implemented | Bundled under `assets/sprite-packs/default/` using `sprite.png` (8x7, 512x512 frames) |
|
||||
| Build/package automation | Implemented (Windows) | `justfile` and `scripts/package_windows.py` generate portable ZIP + SHA256 |
|
||||
| Random backend API tester | Implemented | `scripts/random_backend_tester.py` with `just random-backend-test` and strict variant |
|
||||
| QA/documentation workflow | Implemented | `docs/QA_WORKFLOW.md`, issue/evidence templates, and `scripts/qa_validate.py` with `just qa-validate` |
|
||||
| Shared runtime core | In progress | `sprimo-runtime-core` extracted with shared config/snapshot/API startup and command application |
|
||||
| Tauri alternative frontend | In progress | `sprimo-tauri` now runs runtime-core/API + PixiJS sprite rendering shell, parity work remains |
|
||||
|
||||
@@ -72,6 +72,12 @@ cargo test --workspace
|
||||
just qa-validate
|
||||
```
|
||||
|
||||
Optional runtime/API stress validation:
|
||||
|
||||
```powershell
|
||||
just random-backend-test
|
||||
```
|
||||
|
||||
For runtime behavior issues, include screenshot capture paths in the issue file.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
100
docs/RANDOM_BACKEND_TESTING.md
Normal file
100
docs/RANDOM_BACKEND_TESTING.md
Normal file
@@ -0,0 +1,100 @@
|
||||
# Random Backend API Testing
|
||||
|
||||
Date: 2026-02-13
|
||||
|
||||
## Purpose
|
||||
|
||||
This workflow provides randomized backend-like API traffic against a running Sprimo frontend.
|
||||
It focuses on command endpoints and mixes valid and invalid requests to verify transport and
|
||||
runtime resilience.
|
||||
|
||||
Primary targets:
|
||||
|
||||
- `POST /v1/command`
|
||||
- `POST /v1/commands`
|
||||
|
||||
Supporting checks:
|
||||
|
||||
- `GET /v1/health`
|
||||
- `GET /v1/state` (periodic sampling)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Frontend runtime is already running (`sprimo-app` or `sprimo-tauri`).
|
||||
- Python is available.
|
||||
- Auth token and port are available from config or passed via CLI flags.
|
||||
|
||||
By default, the tester discovers config at:
|
||||
|
||||
- Windows: `%APPDATA%/sprimo/config/config.toml` (legacy fallback: `%APPDATA%/sprimo/config.toml`)
|
||||
- macOS: `~/Library/Application Support/sprimo/config/config.toml`
|
||||
- Linux: `~/.config/sprimo/config/config.toml`
|
||||
|
||||
## Quick Start
|
||||
|
||||
```powershell
|
||||
just random-backend-test
|
||||
```
|
||||
|
||||
Strict mode (non-zero exit if unexpected outcomes appear):
|
||||
|
||||
```powershell
|
||||
just random-backend-test-strict
|
||||
```
|
||||
|
||||
## CLI Examples
|
||||
|
||||
Run against explicit host/port/token:
|
||||
|
||||
```powershell
|
||||
python scripts/random_backend_tester.py --host 127.0.0.1 --port 32145 --token "<token>"
|
||||
```
|
||||
|
||||
Deterministic run with higher invalid traffic:
|
||||
|
||||
```powershell
|
||||
python scripts/random_backend_tester.py --seed 42 --invalid-probability 0.35 --duration-seconds 45
|
||||
```
|
||||
|
||||
Write machine-readable summary:
|
||||
|
||||
```powershell
|
||||
python scripts/random_backend_tester.py --json-summary dist/random-backend-summary.json
|
||||
```
|
||||
|
||||
## Important Flags
|
||||
|
||||
- `--duration-seconds`: total run time
|
||||
- `--interval-ms`: delay between random requests
|
||||
- `--batch-probability`: ratio of `/v1/commands` usage
|
||||
- `--max-batch-size`: max commands per batch request
|
||||
- `--invalid-probability`: inject malformed/invalid payloads
|
||||
- `--unauthorized-probability`: inject auth failures
|
||||
- `--state-sample-every`: periodic authenticated `/v1/state` checks
|
||||
- `--strict`: fail run on unexpected outcomes
|
||||
- `--health-check`: verify API liveness before random traffic
|
||||
|
||||
## Expected Result Pattern
|
||||
|
||||
In mixed mode, typical status distribution includes:
|
||||
|
||||
- `202` for valid command requests
|
||||
- `400` for malformed/invalid payloads
|
||||
- `401` for missing/invalid auth
|
||||
|
||||
Unexpected outcomes that should be investigated:
|
||||
|
||||
- `5xx` responses
|
||||
- repeated transport failures/timeouts
|
||||
- strict mode failures (`unexpected_outcomes > 0`)
|
||||
|
||||
## Evidence Guidance
|
||||
|
||||
When used for issue verification, record:
|
||||
|
||||
- command used (including seed/probabilities)
|
||||
- summary output (status buckets, unexpected outcomes, transport errors)
|
||||
- linked issue file under `issues/issueN.md`
|
||||
|
||||
This test complements `cargo check --workspace`, `cargo test --workspace`, and
|
||||
`just qa-validate`; it does not replace them.
|
||||
@@ -29,6 +29,7 @@ just smoke-win-bevy
|
||||
just build-release-tauri
|
||||
just package-win-tauri
|
||||
just smoke-win-tauri
|
||||
just random-backend-test
|
||||
```
|
||||
|
||||
Compatibility aliases:
|
||||
@@ -58,6 +59,9 @@ Run tests from an unpacked ZIP folder, not from the workspace run.
|
||||
8. Confirm overlay background is transparent (desktop visible behind non-sprite pixels).
|
||||
9. Confirm no magenta matte remains around sprite in default pack.
|
||||
10. Confirm default startup window footprint is reduced (416x416 before runtime pack resize).
|
||||
11. Run randomized backend API interaction and review summary output:
|
||||
- `just random-backend-test`
|
||||
- validate expected mix of `202`/`400`/`401` without process crash.
|
||||
|
||||
## Test Log Template
|
||||
|
||||
@@ -106,6 +110,9 @@ Authoritative workflow: `docs/TAURI_RUNTIME_TESTING.md`.
|
||||
- left-mouse drag moves window
|
||||
- window remains non-resizable
|
||||
- moved position updates runtime snapshot `x/y` and persists after restart
|
||||
8. Run randomized backend API interaction:
|
||||
- `just random-backend-test`
|
||||
- verify command traffic remains stable and runtime stays alive.
|
||||
|
||||
### Packaged Mode (Required Once Tauri Packaging Exists)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user