Upload files to "designs/grel-rs"

2026-04-10 16:45:55 +08:00
parent c367527464
commit a969692d04
1 changed files with 310 additions and 0 deletions
--- a/designs/grel-rs/TECHNICAL_DESIGN.md
+++ b/designs/grel-rs/TECHNICAL_DESIGN.md
@@ -0,0 +1,310 @@
+# 📘 `grel-rs` Technical Design Document
+**Binary:** `grel` | **Repository:** `grel-rs`  
+**Target Platforms:** Linux, Windows (macOS optional)  
+**Core Philosophy:** Forge-agnostic, pacman-familiar, performance-first, secure, extensible via traits.
+
+---
+
+## 1. Overview & Goals
+`grel` (Global/General Release) is a terminal-native, high-performance package manager for downloading and managing binary releases from Git forges. It abstracts provider-specific APIs into a unified resolution/download pipeline, supports HTTP/SOCKS5 proxies, caches optimal DNS endpoints, downloads assets in parallel, and presents a polished, pacman-compatible UX.
+
+**Key Requirements Met:**
+- ✅ Pacman-style CLI (`-S`, `-Syu`, `-Ss`, `--noconfirm`)
+- ✅ Proxy support (HTTP/HTTPS/SOCKS5) with env/config fallback
+- ✅ DNS/IP latency caching + RTT probing for CDN optimization
+- ✅ Parallel asset downloads with multi-progress UI
+- ✅ Interactive asset selection (pacman/artix style) with TTY/CI fallback
+- ✅ Cross-platform (Linux/Windows) with safe extraction & PATH integration
+- ✅ Extensible provider architecture (GitHub, GitLab, Codeberg, Gitea, self-hosted)
+
+---
+
+## 2. Workspace Architecture
+```
+grel-rs/
+├── Cargo.toml                  # Workspace root (resolver = "2")
+├── crates/
+│   ├── grel-cli/               # CLI parsing, TUI orchestration, prompts, progress
+│   ├── grel-core/              # Resolution, versioning, asset matching, upgrade state
+│   ├── grel-providers/         # Forge trait, registry, API implementations (modules)
+│   ├── grel-network/           # HTTP client, proxy routing, IP cache resolver, downloader
+│   ├── grel-cache/             # SQLite state, IP cache, artifact storage, TTL eviction
+│   └── grel-config/            # Layered config (CLI > env > TOML > defaults), migrations
+├── tests/                      # Integration, mock servers, e2e fixtures
+└── scripts/                    # CI, release, benchmark helpers
+```
+
+**Crate Boundaries:**
+| Crate | Owns | Depends On |
+|-------|------|------------|
+| `grel-cli` | CLI, UI, command routing, prompts | `grel-core`, `grel-config`, `dialoguer`, `indicatif` |
+| `grel-core` | Package resolution, asset matching, upgrade logic | `grel-providers`, `grel-cache`, `semver` |
+| `grel-providers` | API clients, pagination, rate-limit handling | `reqwest`, `async-trait`, `chrono` |
+| `grel-network` | Proxy, IP resolver, parallel downloads, streaming | `reqwest`, `hickory-resolver`, `dashmap`, `tokio` |
+| `grel-cache` | SQLite state, IP cache, artifact storage, LRU | `sqlx`, `sha2`, `tar`/`zip`, `directories` |
+| `grel-config` | Settings loading, validation, defaults, migrations | `figment`, `serde`, `toml` |
+
+---
+
+## 3. Core Abstractions & Data Flow
+All forges implement a single normalized contract. `grel-core` parses user input → providers return forge-agnostic structs → `grel-network` downloads blindly.
+
+```rust
+pub enum ForgeType { GitHub, GitLab, Gitea, Codeberg, SelfHosted(String) }
+
+pub struct PackageRef {
+    pub forge: ForgeType,
+    pub owner: String, pub repo: String,
+    pub version: Option<semver::Version>,
+    pub prerelease: bool,
+}
+
+pub struct ResolvedRelease {
+    pub version: semver::Version,
+    pub published_at: chrono::DateTime<Utc>,
+    pub is_prerelease: bool,
+    pub assets: Vec<DownloadAsset>,
+    pub release_notes: Option<String>,
+}
+
+pub struct DownloadAsset {
+    pub filename: String,
+    pub download_url: url::Url,
+    pub size_bytes: u64,
+    pub sha256: Option<String>,
+    pub asset_type: AssetKind, // Binary, Archive, Source, Other
+}
+```
+
+**Flow:**
+```
+CLI → PackageRef → ProviderRegistry → Provider.resolve() → ResolvedRelease
+   → AssetResolver (tiered match + checksum pairing) → SelectedRelease
+   → Network (IP cache → proxy client → parallel download)
+   → Cache (verify sha256 → extract → atomic rename → update state.sqlite)
+   → CLI (progress summary → exit)
+```
+
+---
+
+## 4. CLI Specification & UX Design
+`clap 4` with subcommands. Pacman short flags supported via aliases.
+
+| Command | Aliases | Description |
+|---------|---------|-------------|
+| `grel sync foo/bar` | `S`, `-S` | Download & install latest |
+| `grel sync foo/bar@1.2.3` | `-S` | Pin version |
+| `grel search keyword` | `Ss`, `-Ss` | Search across registered forges |
+| `grel upgrade` | `Syu`, `-Syu` | Refresh + upgrade installed |
+| `grel info foo/bar` | `Si`, `-Si` | Show release metadata |
+| `grel list` | `Q`, `-Q` | List installed packages |
+| `grel remove foo/bar` | `R`, `-R` | Uninstall |
+| `grel clean` | `Sc`, `-Sc` | Purge artifact/metadata cache |
+| `grel path add` | - | Inject `bin/` dir into shell/Windows PATH |
+
+**UX Features:**
+- `tabled` for search/list output
+- `indicatif::MultiProgress` for parallel downloads
+- `owo-colors` for status: 🟢 ✅, 🟡 ⚠️, 🔴 ❌, 🔵 ℹ️
+- `--json` for machine-readable output
+- `--noconfirm` / `-y` skips all prompts
+
+---
+
+## 5. PATH Integration & Post-Install Behavior
+**Problem:** User-space installs aren't in `$PATH` by default. Pacman installs globally (`/usr/bin`).
+
+**Solution:** Dedicated `bin/` directory + explicit shell integration.
+```
+~/.local/share/grel/ (Linux)   →  bin/
+%LOCALAPPDATA%\grel\ (Windows) →  bin\
+```
+
+**Implementation:**
+1. `grel sync` extracts binaries to `~/.local/share/grel/bin/` (or equivalent)
+2. `grel path add` detects shell (`$SHELL`, `$PROFILE`) and appends:
+   ```bash
+   export PATH="$HOME/.local/share/grel/bin:$PATH"  # bash/zsh
+   $env:Path = "$env:LOCALAPPDATA\grel\bin;" + $env:Path  # PowerShell
+   ```
+3. Windows: Uses registry `HKCU\Environment\Path` + broadcasts `WM_SETTINGCHANGE`
+4. First-run hint: `💡 Run grel path add to enable global command access`
+5. **Why not `/usr/local/bin`?** Avoids `sudo`, respects XDG, prevents system package conflicts.
+
+---
+
+## 6. Configuration vs State Management
+**Strict Separation:**
+| Layer | Format | Location | Purpose |
+|-------|--------|----------|---------|
+| **User Config** | `config.toml` | `~/.config/grel/` (Linux) / `%APPDATA%\grel\` (Win) | Proxy, tokens, concurrency, asset preferences |
+| **Program State** | `state.sqlite` | `~/.local/share/grel/db/` | Installed packages, IP cache, ETags, last-checked |
+| **Artifact Cache** | Files | `~/.cache/grel/artifacts/` | Downloaded archives (LRU evicted, 2GB default) |
+
+**`config.toml` Example:**
+```toml
+[general]
+version = 1
+max_concurrent = 4
+proxy = "http://127.0.0.1:7890"
+
+[assets]
+default_selection_policy = "largest"  # first | largest | preferred_format
+prefer_formats = ["*.tar.gz", "*.zip", "*.exe"]
+ignore_patterns = ["*source*", "*.deb", "*.rpm"]
+
+[upgrade]
+check_interval_hours = 6
+max_parallel_checks = 10
+```
+
+**Migrations:**
+- Config: `version` field in TOML. On load, `grel-config` runs migration functions (rename keys, add defaults, warn on breaking changes).
+- SQLite: `sqlx migrate` with timestamped files. Applied automatically on startup. Zero-downtime schema upgrades.
+
+---
+
+## 7. Asset Resolution & Selection Pipeline
+Releases contain multiple formats. `grel` uses **deterministic tiered matching** + **interactive fallback**.
+
+### Tiered Matching (Order Matters)
+1. **Exact:** `{repo}-{version}-{os}-{arch}.{fmt}`
+2. **OS+Arch:** `*{os}*{arch}*.{fmt}`
+3. **OS Only:** `*{os}*.{fmt}`
+4. **Universal Archive:** `*.{tar.gz,tar.xz,zip}`
+5. **Universal Binary:** `*.{exe,elf,dll,dylib}`
+
+### Checksum & Signature Auto-Pairing
+If main asset is `foo-1.0.0-linux-amd64.tar.gz`, `grel` searches for:
+- `foo-1.0.0-linux-amd64.tar.gz.sha256`
+- `sha256sums.txt` (parses line matching filename)
+Automatically downloads & verifies before extraction.
+
+### Interactive Selection (Pacman/Artix Style)
+- Uses `dialoguer::Select` with numbered list
+- Default marker: `(default)` shown next to highest-priority match
+- `--noconfirm` / `-y` skips prompt, uses default
+- **TTY Detection:** If not interactive (CI/pipes), auto-selects default + prints `ℹ️ Non-interactive mode. Using default: <filename>` to stderr
+- Timeout fallback (configurable): auto-selects default if no input within `prompt_timeout_secs`
+
+**CLI Overrides:**
+```bash
+grel sync foo/bar --asset custom.exe
+grel sync foo/bar --platform linux/aarch64
+grel sync foo/bar --list-assets  # Dry-run table
+```
+
+---
+
+## 8. Networking, Proxy & DNS/IP Caching
+### Proxy Support
+- Native via `reqwest::Proxy::all()`, `http()`, `https()`
+- Priority: `--proxy` > `GREL_PROXY` > `http_proxy`/`all_proxy` > `config.toml`
+- Supports `http://user:pass@host`, `socks5://...`, `socks5h://...`
+
+### DNS/IP Caching
+⚠️ **No hardcoded IPs.** CDNs rotate frequently. Instead:
+1. Resolve `A/AAAA` via `hickory-resolver`
+2. Parallel `TcpStream::connect` probes to all IPs on `443`
+3. Store fastest IP + RTT in SQLite with TTL (default `300s`)
+4. Use `reqwest::ClientBuilder::resolve(host, ip)` for routing
+5. Background refresh thread evicts stale entries, pre-warms hot domains
+
+---
+
+## 9. Rate Limiting & `-Syu` Optimization
+GitHub allows 60 req/hr unauthenticated, 5000/hr authenticated. `grel` handles this gracefully:
+
+1. **ETag Caching:** `If-None-Match` → `304 Not Modified` **does not count** against rate limits. Cached ETags stored in `state.sqlite`.
+2. **Smart Polling:** Only recheck packages where `last_checked < now() - check_interval_hours`.
+3. **Rate Limit Awareness:** Parse `X-RateLimit-Remaining` & `X-RateLimit-Reset`. If `< 50`, sleep until reset or queue remaining checks.
+4. **Auth Token:** Strongly recommended. `grel` prompts on first run: `🔑 Set GREL_GITHUB_TOKEN for 5000 req/hr`.
+5. **`--no-api` Fallback:** Skips remote checks, uses local manifest + cache timestamps.
+6. **Parallel Batch Checks:** `max_parallel_checks` limits concurrent API calls to avoid hitting burst limits.
+
+**`-Syu` Flow:**
+```
+Load state → Filter stale → Batch API (ETag) → Parse 200/304 → Build diff → Prompt → Download → Verify → Update
+```
+
+---
+
+## 10. Cross-Platform Implementation Notes
+| Aspect | Linux | Windows | Implementation |
+|--------|-------|---------|----------------|
+| Paths | `~/.local/share/grel` | `%LOCALAPPDATA%\grel` | `directories::ProjectDirs` |
+| Archives | `tar` + `zstd`/`xz`/`gz` | `zip` + `tar` | Feature-gated extraction |
+| Exec Perms | `0o755` | `FILE_ATTRIBUTE_ARCHIVE` | `cfg(unix)` vs `cfg(windows)` |
+| Path Safety | Reject `..`, absolute paths | Same | `sanitize-filename` + strict validation |
+| Executable Suffix | None | `.exe` | Auto-append if `cfg(windows)` & missing |
+| Shell Integration | `.bashrc`, `.zshrc`, fish | `$PROFILE`, Registry | `grel path add` |
+
+---
+
+## 11. Security & Reliability Guarantees
+- ✅ **TLS:** `rustls` only (no native OpenSSL)
+- ✅ **Checksums:** Verify `sha256` before extraction
+- ✅ **Archive Safety:** Reject `..`, absolute paths, symlinks to outside dest
+- ✅ **Atomic Installs:** Download to temp dir → verify → `rename` → register in DB
+- ✅ **No Auto-Execute:** Binaries installed but not run unless invoked by user
+- ✅ **Token Security:** Loaded from env/config, never logged or serialized to stdout
+- ✅ **Graceful Degradation:** Network errors → retry with backoff, rate limits → sleep/queue, missing assets → clear error + suggestions
+
+---
+
+## 12. Dependency Matrix
+```toml
+# Core & Async
+tokio = { version = "1", features = ["full"] }
+futures = "0.3"
+async-trait = "0.1"
+
+# CLI & UX
+clap = { version = "4", features = ["derive", "wrap_help", "string"] }
+indicatif = { version = "0.17", features = ["tokio"] }
+tracing-indicatif = "0.3"
+dialoguer = "0.11"
+tabled = "0.16"
+owo-colors = "4"
+
+# Network & Proxy
+reqwest = { version = "0.12", features = ["rustls-tls", "json", "socks", "stream"] }
+hickory-resolver = "0.24"
+dashmap = "6"
+
+# Cache & Storage
+sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "sqlite"] }
+sha2 = "0.10"
+directories = "5"
+tar = "0.4"
+zip = "2"
+zstd = "0.13"
+xz2 = "0.1"
+bzip2 = "0.5"
+sanitize-filename = "0.5"
+
+# Config & Utils
+figment = { version = "0.10", features = ["toml", "env"] }
+semver = "1"
+url = "2"
+chrono = { version = "0.4", features = ["serde"] }
+serde = { version = "1", features = ["derive"] }
+anyhow = "1"
+thiserror = "2"
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
+```
+*Note: Rust 1.70+ required for `std::io::IsTerminal` (replaces `atty`).*
+
+---
+
+## 13. Testing & CI Strategy
+- **Unit:** Parsing, version comparison, cache TTL math, asset matching tiers
+- **Mocked Providers:** `wiremock` for API responses (200, 304, 403, 429, 500)
+- **Integration:** Download test fixtures → extract → verify structure/checksums
+- **Cross-Platform CI:** `ubuntu-latest`, `windows-latest` (GitHub Actions)
+- **Performance:** `criterion` for DNS resolution + parallel download throughput
+- **Fuzzing:** `cargo-fuzz` on `PackageRef` parser + archive path validation
+- **Non-Interactive Tests:** `echo "" | grel sync foo/bar` must not hang
+
+---