diff --git a/designs/grel-rs/TECHNICAL_DESIGN.md b/designs/grel-rs/TECHNICAL_DESIGN.md index 7d698e0..ca16bda 100644 --- a/designs/grel-rs/TECHNICAL_DESIGN.md +++ b/designs/grel-rs/TECHNICAL_DESIGN.md @@ -1,312 +1,339 @@ -# ๐Ÿ“˜ `grel-rs` Technical Design Document (v3.0) -**Binary:** `grel` | **Repository:** `grel-rs` -**Target Platforms:** Linux, Windows (macOS optional) -**Core Philosophy:** Pure CLI, deterministic asset resolution, pacman/Artix-familiar UX, explicit over implicit, robust upgrade edge-case handling. - ---- - -## 1. Overview & Goals -`grel` (Global/General Release) is a terminal-native, high-performance package manager for downloading and managing binary releases from Git forges. It abstracts provider-specific APIs into a unified resolution pipeline, supports HTTP/SOCKS5 proxies, caches optimal DNS endpoints, downloads assets in parallel, and delivers a transparent, scriptable, pacman-compatible UX. - -**Key Requirements Met:** -- โœ… Pure CLI (no TUI/GUI), standard `std::io` prompts -- โœ… Deterministic asset resolution: **filter โ†’ sort โ†’ explicit prompt** (zero scoring magic) -- โœ… Robust parsing of non-standard release names via `Os`/`Arch` enums + alias mapping -- โœ… Format control: `.deb`/`.rpm`/`.msi`/`.dmg` disabled by default -- โœ… Pacman/Artix-style numbered selection with `[default]`, TTY/CI fallback, `-y`/`--noconfirm` -- โœ… `-Syu` resilience: detects filename renames, handles repo 404s via orphan tracking, explicit migration path -- โœ… Cross-platform PATH integration (`grel path add`), XDG-compliant, no `sudo` -- โœ… Forge-agnostic provider routing (public + self-hosted URLs) - ---- - -## 2. Workspace Architecture -``` -grel-rs/ -โ”œโ”€โ”€ Cargo.toml # Workspace root (resolver = "2") -โ”œโ”€โ”€ crates/ -โ”‚ โ”œโ”€โ”€ grel-cli/ # CLI parsing, pure-text prompts, progress routing -โ”‚ โ”œโ”€โ”€ grel-core/ # Resolution, tokenization, filter/sort pipeline, upgrade state -โ”‚ โ”œโ”€โ”€ grel-providers/ # Forge trait, registry, API implementations (modules) -โ”‚ โ”œโ”€โ”€ grel-network/ # HTTP client, proxy routing, IP cache resolver, parallel downloader -โ”‚ โ”œโ”€โ”€ grel-cache/ # SQLite state, IP cache, artifact storage, TTL eviction -โ”‚ โ””โ”€โ”€ grel-config/ # Layered config, migrations, asset priority matrices -โ”œโ”€โ”€ tests/ # Integration, mock servers, e2e fixtures -โ””โ”€โ”€ scripts/ # CI, release, benchmark helpers -``` - -| Crate | Responsibility | -|-------|----------------| -| `grel-cli` | Subcommand routing, pure CLI prompt loops, `indicatif` progress, `tabled` output | -| `grel-core` | `PackageRef` parsing, `AssetTokens` extraction, deterministic filter/sort, upgrade planning | -| `grel-providers` | `ReleaseProvider` trait, GitHub/GitLab/Gitea/Codeberg modules, self-hosted auto-detection | -| `grel-network` | Proxy chaining, DNS/IP cache resolver, `JoinSet` parallel downloads, streaming extraction | -| `grel-cache` | `state.sqlite` management, ETag/IP cache, LRU artifact eviction, atomic install temp dirs | -| `grel-config` | TOML loading, env/CLI overrides, schema validation, config migrations | - ---- - -## 3. CLI Specification & Pure UX -No TUI frameworks. All interaction uses standard terminal I/O with predictable, pipe-safe behavior. - -### Commands & Aliases -| Command | Aliases | Description | -|---------|---------|-------------| -| `grel sync foo/bar` | `S`, `-S` | Download & install latest | -| `grel sync foo/bar@1.2.3` | `-S` | Pin version | -| `grel search keyword` | `Ss`, `-Ss` | Search across registered forges | -| `grel upgrade` | `Syu`, `-Syu` | Refresh + upgrade installed | -| `grel info foo/bar` | `Si`, `-Si` | Show release metadata | -| `grel list` | `Q`, `-Q` | List installed packages (shows status) | -| `grel remove foo/bar` | `R`, `-R` | Uninstall & clean DB | -| `grel clean` | `Sc`, `-Sc` | Purge artifact/metadata cache | -| `grel path add` | - | Generate shell/registry snippets for `$PATH` | -| `grel migrate old/path new/path` | - | Remap repo path in state DB | - -### Pure CLI Prompt (Artix/Pacman Compatible) -``` -:: 3 compatible asset(s) found for linux/x86_64 - 1) foo-1.0.0-linux-x86_64.tar.gz | 12.4 MB | tar.gz (default) - 2) foo-1.0.0-linux-x86_64-musl.tar.gz | 10.2 MB | tar.gz - 3) foo-1.0.0-linux-amd64.tar.gz | 11.8 MB | tar.gz -:: Select asset to download [1]: _ -``` -- Accepts `1`โ€“`3` or `Enter` (selects `[1]`) -- Invalid input โ†’ `:: Invalid selection. Enter a number [1-3]: ` -- `--noconfirm` / `-y` โ†’ skips prompt, selects `[1]` -- **Non-Interactive Fallback:** `!std::io::stdin().is_terminal()` โ†’ auto-selects `[1]`, prints `โ„น๏ธ Non-interactive mode. Using: ` to `stderr`, never blocks CI/pipes. - ---- - -## 4. Asset Resolution Pipeline (Deterministic) -**No scoring. No fuzzy logic.** Strict filtering โ†’ transparent sorting โ†’ explicit selection. - -### Step 1: Tokenization -Filenames are split on non-alphanumeric characters. Tokens are matched against case-insensitive alias maps for `Os` and `Arch`. -```rust -// foo-v1.2.3-win64-setup.exe โ†’ Os::Windows, Arch::X86_64 -// bar_1.0.0_linux_amd64.tar.gz โ†’ Os::Linux, Arch::X86_64 -// app.Darwin.arm64.zip โ†’ Os::MacOS, Arch::Aarch64 -``` -Unknown tokens fall back to `Os::Unknown("...")` or `Arch::Unknown("...")` โ†’ never crash, just filter out later. - -### Step 2: Strict Filtering -Assets are filtered in fixed order. Failure at any step โ†’ discard. -```rust -1. OS matches target OR is Unknown -2. Arch matches priority list OR fallback allowed (32-bit on 64-bit) -3. Format NOT in `ignore_formats` (deb/rpm/msi/dmg disabled by default) -``` - -### Step 3: Deterministic Sorting -Remaining assets sorted by explicit cascade: -1. `arch_priority` index (lower = better) -2. `prefer_formats` index -3. Lexicographic filename -4. Size descending (tie-breaker) - -### Step 4: Selection -- `0` assets โ†’ `โŒ No compatible assets. Run grel sync --list-assets` -- `1` asset โ†’ Auto-select, log `โ„น๏ธ Selected: ` -- `>1` assets โ†’ Pure CLI numbered prompt (see ยง3) - ---- - -## 5. Platform Enums & Alias Mapping -Strongly-typed, exhaustive, infallible parsing. -```rust -#[derive(Debug, Clone, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Os { Linux, Windows, MacOS, FreeBSD, Android, iOS, Unknown(String) } - -#[derive(Debug, Clone, PartialEq, Eq, Hash)] -#[non_exhaustive] -pub enum Arch { X86_64, Aarch64, I686, ArmV7, ArmV6, Riscv64, S390x, PowerPC64, Unknown(String) } -``` -- `FromStr` implemented with `to_lowercase()` + alias table -- `serde` ready for TOML config -- `Display` outputs canonical names for DB/CLI consistency - ---- - -## 6. Configuration vs State Management -**Strict separation.** Humans edit TOML. Machines manage SQLite. - -### `~/.config/grel/config.toml` -```toml -[general] -version = 1 -max_concurrent = 4 -proxy = "http://127.0.0.1:7890" - -[assets] -ignore_formats = ["*.deb", "*.rpm", "*.msi", "*.dmg", "*.pkg", "*.AppImage"] -prefer_formats = ["*.tar.gz", "*.tar.xz", "*.zip", "*.exe"] -arch_priority = ["x86_64", "aarch64", "x86", "armv7"] -fallback_to_32bit = true -prefer_musl = false - -[migrations] -"legacy/old-tool" = "new-org/old-tool" - -[upgrade] -check_interval_hours = 6 -max_parallel_checks = 10 -``` - -### `state.sqlite` Schema (`installed` table) -```sql -CREATE TABLE installed ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - forge TEXT NOT NULL, -- 'github', 'gitlab', or self-hosted base URL - owner TEXT NOT NULL, - repo TEXT NOT NULL, - version TEXT NOT NULL, -- Strict semver - asset_filename TEXT NOT NULL, -- Exact remote filename used - checksum TEXT, - install_path TEXT NOT NULL, - status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'orphaned', 'migrated')), - orphaned_at INTEGER, -- Unix timestamp when marked unreachable - last_checked INTEGER, -- For smart polling intervals - installed_at INTEGER DEFAULT (strftime('%s', 'now')) -); -CREATE UNIQUE INDEX idx_pkg_unique ON installed(forge, owner, repo); -CREATE INDEX idx_status ON installed(status); -``` - ---- - -## 7. `-Syu` Upgrade Logic & Edge Cases -### 7.1 Filename Change Detection -Authors often rename assets (`x86_64` โ†’ `amd64`, `.tar.gz` โ†’ `.tar.xz`). -1. Load stored `asset_filename` from DB -2. Resolve new release โ†’ filter โ†’ sort โ†’ get `new_best` -3. If `new_best.filename != stored.filename`: - ``` - โš ๏ธ Remote asset renamed: old-name.tar.gz โ†’ new-name.tar.xz - โ„น๏ธ Proceeding with update... - ``` -4. Download โ†’ verify โ†’ extract โ†’ **update DB record** with new filename. Proceeds safely. - -### 7.2 Repository Rename / 404 Handling -1. Provider fetch โ†’ `404` or unreachable -2. Mark `status = 'orphaned'`, set `orphaned_at = now()` -3. Skip in future `-Syu` runs -4. Warn user: - ``` - โš ๏ธ Package unreachable: foo/bar (HTTP 404) - โ„น๏ธ Skipped. Run `grel migrate foo/bar new/path` or `grel remove foo/bar` - ``` -5. **No auto-migration.** Explicit user action required. - -### 7.3 Orphaned Visibility -```bash -$ grel list -:: Installed packages (3 active, 1 orphaned) - ๐ŸŸข bar/fuzz 1.2.3 linux/x86_64 tar.gz - ๐ŸŸข foo/tool 0.9.1 windows/amd64 exe - ๐ŸŸก legacy/old-proj 2.0.0 linux/x86_64 tar.gz [orphaned since 2024-05-01] -``` - ---- - -## 8. Networking, Proxy & DNS/IP Caching -- **Proxy Priority:** `--proxy` > `GREL_PROXY` env > `http_proxy`/`all_proxy` > `config.toml` -- **DNS/IP Cache:** - 1. Resolve `A/AAAA` via `hickory-resolver` - 2. Parallel `TcpStream::connect` probes on `443` - 3. Store fastest IP + RTT in SQLite with `300s` TTL - 4. `reqwest::ClientBuilder::resolve(host, ip)` forces routing -- **Background Refresh:** Idle task pre-warms hot domains, evicts stale entries - ---- - -## 9. Rate Limiting & ETag Optimization -GitHub: 60/hr unauth, 5000/hr auth. `grel` handles gracefully: -1. **ETag Caching:** `If-None-Match` โ†’ `304 Not Modified` **free**. Cached in DB. -2. **Smart Polling:** Only checks packages where `last_checked < now() - check_interval_hours` -3. **Rate Limit Parsing:** Reads `X-RateLimit-Remaining`/`Reset`. If `< 50` โ†’ sleep/queue. -4. **Auth Prompt:** First run suggests `GREL_GITHUB_TOKEN` for 5000/hr. -5. **`--no-api` Fallback:** Skips remote checks, uses local timestamps only. - ---- - -## 10. PATH Integration & Post-Install -- Installs to `~/.local/share/grel/bin/` (Linux) / `%LOCALAPPDATA%\grel\bin\` (Windows) -- `grel path add` prints exact shell/registry snippets: - ```bash - export PATH="$HOME/.local/share/grel/bin:$PATH" - ``` - ```powershell - [Environment]::SetEnvironmentVariable("Path", "$env:LOCALAPPDATA\grel\bin;$env:Path", "User") - ``` -- First run: `๐Ÿ’ก Run: grel path add to enable global command access` -- XDG-compliant, no `sudo`, no system conflicts. - ---- - -## 11. Security & Reliability -- โœ… TLS: `rustls` only (no native OpenSSL) -- โœ… Checksums: `sha256` verification before extraction -- โœ… Archive safety: Reject `..`, absolute paths, external symlinks -- โœ… Atomic installs: Temp dir โ†’ verify โ†’ `rename` โ†’ DB update -- โœ… No auto-exec: Binaries installed but not run unless invoked -- โœ… Graceful degradation: Network errors โ†’ retry/backoff, rate limits โ†’ sleep/queue, missing assets โ†’ clear error + suggestions - ---- - -## 12. Dependency Matrix -```toml -# Core & Async -tokio = { version = "1", features = ["full"] } -futures = "0.3" -async-trait = "0.1" - -# CLI & UX (Pure CLI, no TUI) -clap = { version = "4", features = ["derive", "wrap_help", "string"] } -indicatif = { version = "0.17", features = ["tokio"] } -tracing-indicatif = "0.3" -tabled = "0.16" -owo-colors = "4" - -# Network & Proxy -reqwest = { version = "0.12", features = ["rustls-tls", "json", "socks", "stream"] } -hickory-resolver = "0.24" -dashmap = "6" - -# Cache & Storage -sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "sqlite"] } -sha2 = "0.10" -directories = "5" -tar = "0.4" -zip = "2" -zstd = "0.13" -xz2 = "0.1" -bzip2 = "0.5" -sanitize-filename = "0.5" - -# Config & Utils -figment = { version = "0.10", features = ["toml", "env"] } -semver = "1" -url = "2" -chrono = { version = "0.4", features = ["serde"] } -regex = "1" -serde = { version = "1", features = ["derive"] } -anyhow = "1" -thiserror = "2" -tracing = "0.1" -tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] } -``` - ---- - -## 13. Testing & CI Strategy -- **Unit:** Token extraction from 20+ messy filenames, filter/sort determinism, config parsing, migration logic -- **Mocked Providers:** `wiremock` for 200/304/403/429/500, rate limit header injection -- **Integration:** Download fixtures โ†’ extract โ†’ verify checksums & paths, `grel migrate` DB state changes -- **Non-Interactive:** `echo "" | grel sync foo/bar` must not hang, must log to stderr -- **Cross-Platform CI:** `ubuntu-latest`, `windows-latest` (GitHub Actions) -- **Performance:** `criterion` for DNS/IP cache + parallel download throughput -- **Fuzzing:** `cargo-fuzz` on `AssetTokens::from_filename()` + archive path validation - ---- +# ๐Ÿ“˜ `grel-rs` Technical Design Document +**Binary:** `grel` | **Repository:** `grel-rs` +**Target Platforms:** Linux, Windows (macOS optional) +**Core Philosophy:** Pure CLI, deterministic asset resolution, explicit over implicit, robust upgrade handling, transparent user control. + + +--- + +## 1. Overview & Goals +`grel` is a terminal-native, high-performance release downloader and package manager for Git forges. It abstracts provider APIs into a unified pipeline, supports proxies, caches DNS/IPs for CDN routing, downloads in parallel, and delivers a transparent, scriptable, pacman-compatible UX. + +**Key Requirements Met:** +- โœ… Pure CLI (no TUI/GUI), standard `std::io` prompts & warnings +- โœ… Deterministic resolution: **strict filters โ†’ priority sorting โ†’ explicit policy fallback** (zero scoring) +- โœ… `exclude_keywords` config to block installer/setup/bundle artifacts +- โœ… Warning system for unmanaged or extra-step packages (applies to `ignore_formats` overrides & keyword matches) +- โœ… Configurable `download_dir` for unmanaged packages (defaults to OS `Downloads/`) +- โœ… `default_selection_policy` (`first` | `largest`) as explicit fallback, overridden by detailed flags +- โœ… `-Syu` resilience: filename rename detection, orphan tracking, explicit migration +- โœ… Cross-platform PATH integration, XDG-compliant, no `sudo` + + +--- + +## 2. Workspace Architecture +``` +grel-rs/ +โ”œโ”€โ”€ Cargo.toml # Workspace root (resolver = "2") +โ”œโ”€โ”€ crates/ +โ”‚ โ”œโ”€โ”€ grel-cli/ # CLI parsing, pure-text prompts, progress routing +โ”‚ โ”œโ”€โ”€ grel-core/ # Resolution, tokenization, filter/sort pipeline, upgrade state +โ”‚ โ”œโ”€โ”€ grel-providers/ # Forge trait, registry, API implementations (modules) +โ”‚ โ”œโ”€โ”€ grel-network/ # HTTP client, proxy routing, IP cache resolver, parallel downloader +โ”‚ โ”œโ”€โ”€ grel-cache/ # SQLite state, IP cache, artifact storage, TTL eviction +โ”‚ โ””โ”€โ”€ grel-config/ # Layered config, migrations, asset priority matrices +โ”œโ”€โ”€ tests/ # Integration, mock servers, e2e fixtures +โ””โ”€โ”€ scripts/ # CI, release, benchmark helpers +``` + +| Crate | Responsibility | +|-------|----------------| +| `grel-cli` | Subcommand routing, pure CLI prompt loops, `indicatif` progress, `tabled` output | +| `grel-core` | `PackageRef` parsing, `AssetTokens` extraction, deterministic filter/sort, upgrade planning | +| `grel-providers` | `ReleaseProvider` trait, GitHub/GitLab/Gitea/Codeberg modules, self-hosted auto-detection | +| `grel-network` | Proxy chaining, DNS/IP cache resolver, `JoinSet` parallel downloads, streaming extraction | +| `grel-cache` | `state.sqlite` management, ETag/IP cache, LRU artifact eviction, atomic install temp dirs | +| `grel-config` | TOML loading, env/CLI overrides, schema validation, config migrations | + +--- + +## 3. CLI Specification & Warning Flow +All interaction uses standard terminal I/O. Unmanaged packages trigger explicit warnings & confirmation. + +### Warning & Confirmation Flow +When an asset matches `exclude_keywords` or falls under an overridden `ignore_formats`: +``` +โš ๏ธ Asset "foo-setup-1.0.0.exe" matches excluded keyword "setup". +โ„น๏ธ grel cannot manage installers directly. Package will download to ~/Downloads/. +:: Proceed with download? [y/N]: _ +``` +- `y`/`Enter` โ†’ Downloads to `download_dir`, marks `is_managed = false` in DB +- `N`/`n`/`Esc` โ†’ Aborts sync for this package, continues with others +- `--noconfirm` / `-y` โ†’ Auto-accepts, prints `โ„น๏ธ Non-interactive: accepted unmanaged asset` to `stderr` +- **Never blocks CI/pipes:** `!std::io::stdin().is_terminal()` โ†’ auto-accepts, logs to `stderr` + +--- + +## 4. Asset Resolution Pipeline (Deterministic) +**No scoring. No fuzzy logic.** Strict filtering โ†’ transparent priority โ†’ policy fallback. + +### Precedence Rules (Strict โ†’ Override โ†’ Fallback) +| Priority | Mechanism | Override Capability | +|----------|-----------|---------------------| +| 1๏ธโƒฃ | OS/Arch exact match or `Unknown` | None | +| 2๏ธโƒฃ | `exclude_keywords` filter | None (hard block unless CLI `--allow-keyword`) | +| 3๏ธโƒฃ | `ignore_formats` filter | Overridable via CLI/config | +| 4๏ธโƒฃ | `arch_priority` index | Overrides `default_selection_policy` | +| 5๏ธโƒฃ | `prefer_formats` index | Overrides `default_selection_policy` | +| 6๏ธโƒฃ | `default_selection_policy` | **Only applies to remaining ties** | + +### Step 1: Strict Filtering +```rust +assets.iter() + .map(|a| AssetTokens::from_filename(&a.filename)) + .filter(|t| t.os == target.os || t.os == Os::Unknown) + .filter(|t| !keyword_excluded(t, &config.exclude_keywords)) + .filter(|t| arch_matches_priority(t, &config.arch_priority, config.fallback_to_32bit)) + .filter(|t| !format_ignored(t, &config.ignore_formats)) + .collect() +``` + +### Step 2: Deterministic Sorting +Sorted by explicit cascade: +1. `arch_priority` index +2. `prefer_formats` index +3. Lexicographic filename +4. Size descending + +### Step 3: Policy Fallback (`default_selection_policy`) +Only applied if `>1` asset survives sorting and remains tied. +- `first` โ†’ picks top of sorted list +- `largest` โ†’ picks by `size_bytes` descending +- **Never overrides** arch/format priority or keyword filters. + +### Step 4: Selection & Warning +- `0` โ†’ `โŒ No compatible assets` +- `1` โ†’ Check if unmanaged โ†’ warn/confirm โ†’ auto-select or prompt +- `>1` โ†’ Pure CLI numbered prompt โ†’ warn/confirm if unmanaged + +--- + +## 5. Platform Enums & Alias Mapping +Strongly-typed, exhaustive, infallible parsing. +```rust +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum Os { Linux, Windows, MacOS, FreeBSD, Android, iOS, Unknown(String) } + +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum Arch { X86_64, Aarch64, I686, ArmV7, ArmV6, Riscv64, S390x, PowerPC64, Unknown(String) } +``` +- `FromStr` implemented with `to_lowercase()` + alias table +- `serde` ready for TOML config +- `Display` outputs canonical names for DB/CLI consistency + +--- + +## 6. Configuration Structure (Updated) +Strict separation of human-editable TOML and machine-managed SQLite. + +```toml +# ~/.config/grel/config.toml +[general] +version = 1 +max_concurrent = 4 +proxy = "http://127.0.0.1:7890" + +[assets] +default_selection_policy = "largest" # first | largest (tiebreaker only) +exclude_keywords = ["setup", "installer", "portable", "bundle", "nupkg"] +ignore_formats = ["*.deb", "*.rpm", "*.msi", "*.dmg", "*.pkg", "*.AppImage"] +prefer_formats = ["*.tar.gz", "*.tar.xz", "*.zip", "*.exe"] +arch_priority = ["x86_64", "aarch64", "x86", "armv7"] +fallback_to_32bit = true +prefer_musl = false + +[paths] +install_root = "~/.local/share/grel" +bin_dir = "~/.local/share/grel/bin" +download_dir = "~/Downloads" # Fallback for unmanaged/extra-op packages + +[upgrade] +check_interval_hours = 6 +max_parallel_checks = 10 + +[migrations] +"legacy/old-tool" = "new-org/old-tool" +``` + +--- + +## 7. State Database Schema (`state.sqlite`) +```sql +CREATE TABLE installed ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + + forge TEXT NOT NULL, + owner TEXT NOT NULL, + repo TEXT NOT NULL, + version TEXT NOT NULL, + asset_filename TEXT NOT NULL, + checksum TEXT, + install_path TEXT NOT NULL, -- Actual path (bin/ or download_dir) + is_managed BOOLEAN NOT NULL DEFAULT 1, + status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'orphaned', 'migrated')), + orphaned_at INTEGER, + last_checked INTEGER, + installed_at INTEGER DEFAULT (strftime('%s', 'now')) +); +CREATE UNIQUE INDEX idx_pkg_unique ON installed(forge, owner, repo); +CREATE INDEX idx_status ON installed(status); +``` +- `is_managed`: `true` = auto-extracted/linked to `bin/`; `false` = left in `download_dir` +- `install_path`: Stores actual destination for accurate cleanup/migration + +--- + +## 8. Download Path & Installation Behavior +| Asset Type | Destination | Management | +|------------|-------------|------------| +| Standard binary/archive (`.tar.gz`, `.zip`, `.exe`) | `bin_dir/` (Linux) / `bin\` (Win) | โœ… Managed (extracted, checksummed, linked) | +| Unmanaged/Extra-op (`.msi`, `.deb`, matched keywords) | `download_dir` (default: `~/Downloads`) | โš ๏ธ Download-only, no extraction/execution | +| CLI Override (`--output-dir ~/tmp`) | User-specified path | โœ… Respected regardless of type | + +**Cross-Platform Default Resolution:** +```rust +fn default_download_dir() -> PathBuf { + directories::UserDirs::new() + .map(|d| d.download_dir().clone()) + .unwrap_or_else(|| std::env::temp_dir().join("grel-downloads")) +} +``` +--- + +## 9. `-Syu` Upgrade Logic & Edge Cases +### 9.1 Filename Change Detection +Authors often rename assets (`x86_64` โ†’ `amd64`, `.tar.gz` โ†’ `.tar.xz`). +1. Load stored `asset_filename` from DB +2. Resolve new release โ†’ filter โ†’ sort โ†’ get `new_best` +3. If `new_best.filename != stored.filename`: + ``` + โš ๏ธ Remote asset renamed: old-name.tar.gz โ†’ new-name.tar.xz + โ„น๏ธ Proceeding with update... + ``` +4. Download โ†’ verify โ†’ extract โ†’ **update DB record** with new filename. Proceeds safely. + +### 9.2 Repository Rename / 404 Handling +1. Provider fetch โ†’ `404` or unreachable +2. Mark `status = 'orphaned'`, set `orphaned_at = now()` +3. Skip in future `-Syu` runs +4. Warn user: + ``` + โš ๏ธ Package unreachable: foo/bar (HTTP 404) + โ„น๏ธ Skipped. Run `grel migrate foo/bar new/path` or `grel remove foo/bar` + ``` +5. **No auto-migration.** Explicit user action required. + +### 9.3 Orphaned Visibility +```bash +$ grel list +:: Installed packages (3 active, 1 orphaned) + ๐ŸŸข bar/fuzz 1.2.3 linux/x86_64 tar.gz + ๐ŸŸข foo/tool 0.9.1 windows/amd64 exe + ๐ŸŸก legacy/old-proj 2.0.0 linux/x86_64 tar.gz [orphaned since 2024-05-01] +``` + +--- + +## 10. Networking, Proxy & DNS/IP Caching +- **Proxy Priority:** `--proxy` > `GREL_PROXY` env > `http_proxy`/`all_proxy` > `config.toml` +- **DNS/IP Cache:** + 1. Resolve `A/AAAA` via `hickory-resolver` + 2. Parallel `TcpStream::connect` probes on `443` + 3. Store fastest IP + RTT in SQLite with `300s` TTL + 4. `reqwest::ClientBuilder::resolve(host, ip)` forces routing +- **Background Refresh:** Idle task pre-warms hot domains, evicts stale entries + +--- + +## 11. Rate Limiting & ETag Optimization +GitHub: 60/hr unauth, 5000/hr auth. `grel` handles gracefully: +1. **ETag Caching:** `If-None-Match` โ†’ `304 Not Modified` **free**. Cached in DB. +2. **Smart Polling:** Only checks packages where `last_checked < now() - check_interval_hours` +3. **Rate Limit Parsing:** Reads `X-RateLimit-Remaining`/`Reset`. If `< 50` โ†’ sleep/queue. +4. **Auth Prompt:** First run suggests `GREL_GITHUB_TOKEN` for 5000/hr. +5. **`--no-api` Fallback:** Skips remote checks, uses local timestamps only. + +--- + +## 12. PATH Integration & Post-Install +- Installs to `~/.local/share/grel/bin/` (Linux) / `%LOCALAPPDATA%\grel\bin\` (Windows) +- `grel path add` prints exact shell/registry snippets: + ```bash + export PATH="$HOME/.local/share/grel/bin:$PATH" + ``` + ```powershell + [Environment]::SetEnvironmentVariable("Path", "$env:LOCALAPPDATA\grel\bin;$env:Path", "User") + ``` +- First run: `๐Ÿ’ก Run: grel path add to enable global command access` +- XDG-compliant, no `sudo`, no system conflicts. + +--- + +## 13. Security & Reliability +- โœ… TLS: `rustls` only +- โœ… Checksums: `sha256` before extraction (managed packages only) +- โœ… Archive safety: Reject `..`, absolute paths, external symlinks +- โœ… Atomic installs: Temp dir โ†’ verify โ†’ `rename` โ†’ DB update +- โœ… Unmanaged warnings: Explicit user consent required (unless `--noconfirm`) +- โœ… Graceful degradation: Network errors โ†’ retry/backoff, rate limits โ†’ sleep/queue, missing assets โ†’ clear error + suggestions + +--- + +## 14. Dependency Matrix +```toml +# Core & Async +tokio = { version = "1", features = ["full"] } +futures = "0.3" +async-trait = "0.1" + +# CLI & UX (Pure CLI, no TUI) +clap = { version = "4", features = ["derive", "wrap_help", "string"] } +indicatif = { version = "0.17", features = ["tokio"] } +tracing-indicatif = "0.3" +tabled = "0.16" +owo-colors = "4" + +# Network & Proxy +reqwest = { version = "0.12", features = ["rustls-tls", "json", "socks", "stream"] } +hickory-resolver = "0.24" +dashmap = "6" + +# Cache & Storage +sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "sqlite"] } +sha2 = "0.10" +directories = "5" +tar = "0.4" +zip = "2" +zstd = "0.13" +xz2 = "0.1" +bzip2 = "0.5" +sanitize-filename = "0.5" + +# Config & Utils +figment = { version = "0.10", features = ["toml", "env"] } +semver = "1" +url = "2" +chrono = { version = "0.4", features = ["serde"] } +regex = "1" +serde = { version = "1", features = ["derive"] } +anyhow = "1" +thiserror = "2" +tracing = "0.1" +tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] } +``` + +--- + +## 15. Testing & CI Strategy +- **Unit:** Keyword exclusion, policy fallback precedence, filter/sort determinism, config parsing +- **Mocked Providers:** `wiremock` for 200/304/403/429/500, rate limit headers +- **Integration:** Unmanaged package download โ†’ `download_dir` verification, `grel migrate` state changes +- **Non-Interactive:** `echo "" | grel sync foo/bar` must auto-accept with stderr warning +- **Cross-Platform CI:** `ubuntu-latest`, `windows-latest` +- **Performance:** `criterion` for DNS/IP cache + parallel download throughput +- **Fuzzing:** `cargo-fuzz` on `AssetTokens::from_filename()` + archive path validation + +---