101 lines
2.6 KiB
Markdown
101 lines
2.6 KiB
Markdown
# Random Backend API Testing
|
|
|
|
Date: 2026-02-13
|
|
|
|
## Purpose
|
|
|
|
This workflow provides randomized backend-like API traffic against a running Sprimo frontend.
|
|
It focuses on command endpoints and mixes valid and invalid requests to verify transport and
|
|
runtime resilience.
|
|
|
|
Primary targets:
|
|
|
|
- `POST /v1/command`
|
|
- `POST /v1/commands`
|
|
|
|
Supporting checks:
|
|
|
|
- `GET /v1/health`
|
|
- `GET /v1/state` (periodic sampling)
|
|
|
|
## Prerequisites
|
|
|
|
- Frontend runtime is already running (`sprimo-app` or `sprimo-tauri`).
|
|
- Python is available.
|
|
- Auth token and port are available from config or passed via CLI flags.
|
|
|
|
By default, the tester discovers config at:
|
|
|
|
- Windows: `%APPDATA%/sprimo/config/config.toml` (legacy fallback: `%APPDATA%/sprimo/config.toml`)
|
|
- macOS: `~/Library/Application Support/sprimo/config/config.toml`
|
|
- Linux: `~/.config/sprimo/config/config.toml`
|
|
|
|
## Quick Start
|
|
|
|
```powershell
|
|
just random-backend-test
|
|
```
|
|
|
|
Strict mode (non-zero exit if unexpected outcomes appear):
|
|
|
|
```powershell
|
|
just random-backend-test-strict
|
|
```
|
|
|
|
## CLI Examples
|
|
|
|
Run against explicit host/port/token:
|
|
|
|
```powershell
|
|
python scripts/random_backend_tester.py --host 127.0.0.1 --port 32145 --token "<token>"
|
|
```
|
|
|
|
Deterministic run with higher invalid traffic:
|
|
|
|
```powershell
|
|
python scripts/random_backend_tester.py --seed 42 --invalid-probability 0.35 --duration-seconds 45
|
|
```
|
|
|
|
Write machine-readable summary:
|
|
|
|
```powershell
|
|
python scripts/random_backend_tester.py --json-summary dist/random-backend-summary.json
|
|
```
|
|
|
|
## Important Flags
|
|
|
|
- `--duration-seconds`: total run time
|
|
- `--interval-ms`: delay between random requests
|
|
- `--batch-probability`: ratio of `/v1/commands` usage
|
|
- `--max-batch-size`: max commands per batch request
|
|
- `--invalid-probability`: inject malformed/invalid payloads
|
|
- `--unauthorized-probability`: inject auth failures
|
|
- `--state-sample-every`: periodic authenticated `/v1/state` checks
|
|
- `--strict`: fail run on unexpected outcomes
|
|
- `--health-check`: verify API liveness before random traffic
|
|
|
|
## Expected Result Pattern
|
|
|
|
In mixed mode, typical status distribution includes:
|
|
|
|
- `202` for valid command requests
|
|
- `400` for malformed/invalid payloads
|
|
- `401` for missing/invalid auth
|
|
|
|
Unexpected outcomes that should be investigated:
|
|
|
|
- `5xx` responses
|
|
- repeated transport failures/timeouts
|
|
- strict mode failures (`unexpected_outcomes > 0`)
|
|
|
|
## Evidence Guidance
|
|
|
|
When used for issue verification, record:
|
|
|
|
- command used (including seed/probabilities)
|
|
- summary output (status buckets, unexpected outcomes, transport errors)
|
|
- linked issue file under `issues/issueN.md`
|
|
|
|
This test complements `cargo check --workspace`, `cargo test --workspace`, and
|
|
`just qa-validate`; it does not replace them.
|