Upload files to "designs/grel-rs"
This commit is contained in:
@@ -1,21 +1,22 @@
|
|||||||
# 📘 `grel-rs` Technical Design Document
|
# 📘 `grel-rs` Technical Design Document (v3.0)
|
||||||
**Binary:** `grel` | **Repository:** `grel-rs`
|
**Binary:** `grel` | **Repository:** `grel-rs`
|
||||||
**Target Platforms:** Linux, Windows (macOS optional)
|
**Target Platforms:** Linux, Windows (macOS optional)
|
||||||
**Core Philosophy:** Forge-agnostic, pacman-familiar, performance-first, secure, extensible via traits.
|
**Core Philosophy:** Pure CLI, deterministic asset resolution, pacman/Artix-familiar UX, explicit over implicit, robust upgrade edge-case handling.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 1. Overview & Goals
|
## 1. Overview & Goals
|
||||||
`grel` (Global/General Release) is a terminal-native, high-performance package manager for downloading and managing binary releases from Git forges. It abstracts provider-specific APIs into a unified resolution/download pipeline, supports HTTP/SOCKS5 proxies, caches optimal DNS endpoints, downloads assets in parallel, and presents a polished, pacman-compatible UX.
|
`grel` (Global/General Release) is a terminal-native, high-performance package manager for downloading and managing binary releases from Git forges. It abstracts provider-specific APIs into a unified resolution pipeline, supports HTTP/SOCKS5 proxies, caches optimal DNS endpoints, downloads assets in parallel, and delivers a transparent, scriptable, pacman-compatible UX.
|
||||||
|
|
||||||
**Key Requirements Met:**
|
**Key Requirements Met:**
|
||||||
- ✅ Pacman-style CLI (`-S`, `-Syu`, `-Ss`, `--noconfirm`)
|
- ✅ Pure CLI (no TUI/GUI), standard `std::io` prompts
|
||||||
- ✅ Proxy support (HTTP/HTTPS/SOCKS5) with env/config fallback
|
- ✅ Deterministic asset resolution: **filter → sort → explicit prompt** (zero scoring magic)
|
||||||
- ✅ DNS/IP latency caching + RTT probing for CDN optimization
|
- ✅ Robust parsing of non-standard release names via `Os`/`Arch` enums + alias mapping
|
||||||
- ✅ Parallel asset downloads with multi-progress UI
|
- ✅ Format control: `.deb`/`.rpm`/`.msi`/`.dmg` disabled by default
|
||||||
- ✅ Interactive asset selection (pacman/artix style) with TTY/CI fallback
|
- ✅ Pacman/Artix-style numbered selection with `[default]`, TTY/CI fallback, `-y`/`--noconfirm`
|
||||||
- ✅ Cross-platform (Linux/Windows) with safe extraction & PATH integration
|
- ✅ `-Syu` resilience: detects filename renames, handles repo 404s via orphan tracking, explicit migration path
|
||||||
- ✅ Extensible provider architecture (GitHub, GitLab, Codeberg, Gitea, self-hosted)
|
- ✅ Cross-platform PATH integration (`grel path add`), XDG-compliant, no `sudo`
|
||||||
|
- ✅ Forge-agnostic provider routing (public + self-hosted URLs)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -24,72 +25,31 @@
|
|||||||
grel-rs/
|
grel-rs/
|
||||||
├── Cargo.toml # Workspace root (resolver = "2")
|
├── Cargo.toml # Workspace root (resolver = "2")
|
||||||
├── crates/
|
├── crates/
|
||||||
│ ├── grel-cli/ # CLI parsing, TUI orchestration, prompts, progress
|
│ ├── grel-cli/ # CLI parsing, pure-text prompts, progress routing
|
||||||
│ ├── grel-core/ # Resolution, versioning, asset matching, upgrade state
|
│ ├── grel-core/ # Resolution, tokenization, filter/sort pipeline, upgrade state
|
||||||
│ ├── grel-providers/ # Forge trait, registry, API implementations (modules)
|
│ ├── grel-providers/ # Forge trait, registry, API implementations (modules)
|
||||||
│ ├── grel-network/ # HTTP client, proxy routing, IP cache resolver, downloader
|
│ ├── grel-network/ # HTTP client, proxy routing, IP cache resolver, parallel downloader
|
||||||
│ ├── grel-cache/ # SQLite state, IP cache, artifact storage, TTL eviction
|
│ ├── grel-cache/ # SQLite state, IP cache, artifact storage, TTL eviction
|
||||||
│ └── grel-config/ # Layered config (CLI > env > TOML > defaults), migrations
|
│ └── grel-config/ # Layered config, migrations, asset priority matrices
|
||||||
├── tests/ # Integration, mock servers, e2e fixtures
|
├── tests/ # Integration, mock servers, e2e fixtures
|
||||||
└── scripts/ # CI, release, benchmark helpers
|
└── scripts/ # CI, release, benchmark helpers
|
||||||
```
|
```
|
||||||
|
|
||||||
**Crate Boundaries:**
|
| Crate | Responsibility |
|
||||||
| Crate | Owns | Depends On |
|
|-------|----------------|
|
||||||
|-------|------|------------|
|
| `grel-cli` | Subcommand routing, pure CLI prompt loops, `indicatif` progress, `tabled` output |
|
||||||
| `grel-cli` | CLI, UI, command routing, prompts | `grel-core`, `grel-config`, `dialoguer`, `indicatif` |
|
| `grel-core` | `PackageRef` parsing, `AssetTokens` extraction, deterministic filter/sort, upgrade planning |
|
||||||
| `grel-core` | Package resolution, asset matching, upgrade logic | `grel-providers`, `grel-cache`, `semver` |
|
| `grel-providers` | `ReleaseProvider` trait, GitHub/GitLab/Gitea/Codeberg modules, self-hosted auto-detection |
|
||||||
| `grel-providers` | API clients, pagination, rate-limit handling | `reqwest`, `async-trait`, `chrono` |
|
| `grel-network` | Proxy chaining, DNS/IP cache resolver, `JoinSet` parallel downloads, streaming extraction |
|
||||||
| `grel-network` | Proxy, IP resolver, parallel downloads, streaming | `reqwest`, `hickory-resolver`, `dashmap`, `tokio` |
|
| `grel-cache` | `state.sqlite` management, ETag/IP cache, LRU artifact eviction, atomic install temp dirs |
|
||||||
| `grel-cache` | SQLite state, IP cache, artifact storage, LRU | `sqlx`, `sha2`, `tar`/`zip`, `directories` |
|
| `grel-config` | TOML loading, env/CLI overrides, schema validation, config migrations |
|
||||||
| `grel-config` | Settings loading, validation, defaults, migrations | `figment`, `serde`, `toml` |
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 3. Core Abstractions & Data Flow
|
## 3. CLI Specification & Pure UX
|
||||||
All forges implement a single normalized contract. `grel-core` parses user input → providers return forge-agnostic structs → `grel-network` downloads blindly.
|
No TUI frameworks. All interaction uses standard terminal I/O with predictable, pipe-safe behavior.
|
||||||
|
|
||||||
```rust
|
|
||||||
pub enum ForgeType { GitHub, GitLab, Gitea, Codeberg, SelfHosted(String) }
|
|
||||||
|
|
||||||
pub struct PackageRef {
|
|
||||||
pub forge: ForgeType,
|
|
||||||
pub owner: String, pub repo: String,
|
|
||||||
pub version: Option<semver::Version>,
|
|
||||||
pub prerelease: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
pub struct ResolvedRelease {
|
|
||||||
pub version: semver::Version,
|
|
||||||
pub published_at: chrono::DateTime<Utc>,
|
|
||||||
pub is_prerelease: bool,
|
|
||||||
pub assets: Vec<DownloadAsset>,
|
|
||||||
pub release_notes: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
pub struct DownloadAsset {
|
|
||||||
pub filename: String,
|
|
||||||
pub download_url: url::Url,
|
|
||||||
pub size_bytes: u64,
|
|
||||||
pub sha256: Option<String>,
|
|
||||||
pub asset_type: AssetKind, // Binary, Archive, Source, Other
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Flow:**
|
|
||||||
```
|
|
||||||
CLI → PackageRef → ProviderRegistry → Provider.resolve() → ResolvedRelease
|
|
||||||
→ AssetResolver (tiered match + checksum pairing) → SelectedRelease
|
|
||||||
→ Network (IP cache → proxy client → parallel download)
|
|
||||||
→ Cache (verify sha256 → extract → atomic rename → update state.sqlite)
|
|
||||||
→ CLI (progress summary → exit)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. CLI Specification & UX Design
|
|
||||||
`clap 4` with subcommands. Pacman short flags supported via aliases.
|
|
||||||
|
|
||||||
|
### Commands & Aliases
|
||||||
| Command | Aliases | Description |
|
| Command | Aliases | Description |
|
||||||
|---------|---------|-------------|
|
|---------|---------|-------------|
|
||||||
| `grel sync foo/bar` | `S`, `-S` | Download & install latest |
|
| `grel sync foo/bar` | `S`, `-S` | Download & install latest |
|
||||||
@@ -97,51 +57,82 @@ CLI → PackageRef → ProviderRegistry → Provider.resolve() → ResolvedRelea
|
|||||||
| `grel search keyword` | `Ss`, `-Ss` | Search across registered forges |
|
| `grel search keyword` | `Ss`, `-Ss` | Search across registered forges |
|
||||||
| `grel upgrade` | `Syu`, `-Syu` | Refresh + upgrade installed |
|
| `grel upgrade` | `Syu`, `-Syu` | Refresh + upgrade installed |
|
||||||
| `grel info foo/bar` | `Si`, `-Si` | Show release metadata |
|
| `grel info foo/bar` | `Si`, `-Si` | Show release metadata |
|
||||||
| `grel list` | `Q`, `-Q` | List installed packages |
|
| `grel list` | `Q`, `-Q` | List installed packages (shows status) |
|
||||||
| `grel remove foo/bar` | `R`, `-R` | Uninstall |
|
| `grel remove foo/bar` | `R`, `-R` | Uninstall & clean DB |
|
||||||
| `grel clean` | `Sc`, `-Sc` | Purge artifact/metadata cache |
|
| `grel clean` | `Sc`, `-Sc` | Purge artifact/metadata cache |
|
||||||
| `grel path add` | - | Inject `bin/` dir into shell/Windows PATH |
|
| `grel path add` | - | Generate shell/registry snippets for `$PATH` |
|
||||||
|
| `grel migrate old/path new/path` | - | Remap repo path in state DB |
|
||||||
|
|
||||||
**UX Features:**
|
### Pure CLI Prompt (Artix/Pacman Compatible)
|
||||||
- `tabled` for search/list output
|
```
|
||||||
- `indicatif::MultiProgress` for parallel downloads
|
:: 3 compatible asset(s) found for linux/x86_64
|
||||||
- `owo-colors` for status: 🟢 ✅, 🟡 ⚠️, 🔴 ❌, 🔵 ℹ️
|
1) foo-1.0.0-linux-x86_64.tar.gz | 12.4 MB | tar.gz (default)
|
||||||
- `--json` for machine-readable output
|
2) foo-1.0.0-linux-x86_64-musl.tar.gz | 10.2 MB | tar.gz
|
||||||
- `--noconfirm` / `-y` skips all prompts
|
3) foo-1.0.0-linux-amd64.tar.gz | 11.8 MB | tar.gz
|
||||||
|
:: Select asset to download [1]: _
|
||||||
|
```
|
||||||
|
- Accepts `1`–`3` or `Enter` (selects `[1]`)
|
||||||
|
- Invalid input → `:: Invalid selection. Enter a number [1-3]: `
|
||||||
|
- `--noconfirm` / `-y` → skips prompt, selects `[1]`
|
||||||
|
- **Non-Interactive Fallback:** `!std::io::stdin().is_terminal()` → auto-selects `[1]`, prints `ℹ️ Non-interactive mode. Using: <filename>` to `stderr`, never blocks CI/pipes.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 5. PATH Integration & Post-Install Behavior
|
## 4. Asset Resolution Pipeline (Deterministic)
|
||||||
**Problem:** User-space installs aren't in `$PATH` by default. Pacman installs globally (`/usr/bin`).
|
**No scoring. No fuzzy logic.** Strict filtering → transparent sorting → explicit selection.
|
||||||
|
|
||||||
**Solution:** Dedicated `bin/` directory + explicit shell integration.
|
### Step 1: Tokenization
|
||||||
|
Filenames are split on non-alphanumeric characters. Tokens are matched against case-insensitive alias maps for `Os` and `Arch`.
|
||||||
|
```rust
|
||||||
|
// foo-v1.2.3-win64-setup.exe → Os::Windows, Arch::X86_64
|
||||||
|
// bar_1.0.0_linux_amd64.tar.gz → Os::Linux, Arch::X86_64
|
||||||
|
// app.Darwin.arm64.zip → Os::MacOS, Arch::Aarch64
|
||||||
```
|
```
|
||||||
~/.local/share/grel/ (Linux) → bin/
|
Unknown tokens fall back to `Os::Unknown("...")` or `Arch::Unknown("...")` → never crash, just filter out later.
|
||||||
%LOCALAPPDATA%\grel\ (Windows) → bin\
|
|
||||||
|
### Step 2: Strict Filtering
|
||||||
|
Assets are filtered in fixed order. Failure at any step → discard.
|
||||||
|
```rust
|
||||||
|
1. OS matches target OR is Unknown
|
||||||
|
2. Arch matches priority list OR fallback allowed (32-bit on 64-bit)
|
||||||
|
3. Format NOT in `ignore_formats` (deb/rpm/msi/dmg disabled by default)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Implementation:**
|
### Step 3: Deterministic Sorting
|
||||||
1. `grel sync` extracts binaries to `~/.local/share/grel/bin/` (or equivalent)
|
Remaining assets sorted by explicit cascade:
|
||||||
2. `grel path add` detects shell (`$SHELL`, `$PROFILE`) and appends:
|
1. `arch_priority` index (lower = better)
|
||||||
```bash
|
2. `prefer_formats` index
|
||||||
export PATH="$HOME/.local/share/grel/bin:$PATH" # bash/zsh
|
3. Lexicographic filename
|
||||||
$env:Path = "$env:LOCALAPPDATA\grel\bin;" + $env:Path # PowerShell
|
4. Size descending (tie-breaker)
|
||||||
|
|
||||||
|
### Step 4: Selection
|
||||||
|
- `0` assets → `❌ No compatible assets. Run grel sync --list-assets`
|
||||||
|
- `1` asset → Auto-select, log `ℹ️ Selected: <filename>`
|
||||||
|
- `>1` assets → Pure CLI numbered prompt (see §3)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Platform Enums & Alias Mapping
|
||||||
|
Strongly-typed, exhaustive, infallible parsing.
|
||||||
|
```rust
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||||
|
#[non_exhaustive]
|
||||||
|
pub enum Os { Linux, Windows, MacOS, FreeBSD, Android, iOS, Unknown(String) }
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||||
|
#[non_exhaustive]
|
||||||
|
pub enum Arch { X86_64, Aarch64, I686, ArmV7, ArmV6, Riscv64, S390x, PowerPC64, Unknown(String) }
|
||||||
```
|
```
|
||||||
3. Windows: Uses registry `HKCU\Environment\Path` + broadcasts `WM_SETTINGCHANGE`
|
- `FromStr` implemented with `to_lowercase()` + alias table
|
||||||
4. First-run hint: `💡 Run grel path add to enable global command access`
|
- `serde` ready for TOML config
|
||||||
5. **Why not `/usr/local/bin`?** Avoids `sudo`, respects XDG, prevents system package conflicts.
|
- `Display` outputs canonical names for DB/CLI consistency
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Configuration vs State Management
|
## 6. Configuration vs State Management
|
||||||
**Strict Separation:**
|
**Strict separation.** Humans edit TOML. Machines manage SQLite.
|
||||||
| Layer | Format | Location | Purpose |
|
|
||||||
|-------|--------|----------|---------|
|
|
||||||
| **User Config** | `config.toml` | `~/.config/grel/` (Linux) / `%APPDATA%\grel\` (Win) | Proxy, tokens, concurrency, asset preferences |
|
|
||||||
| **Program State** | `state.sqlite` | `~/.local/share/grel/db/` | Installed packages, IP cache, ETags, last-checked |
|
|
||||||
| **Artifact Cache** | Files | `~/.cache/grel/artifacts/` | Downloaded archives (LRU evicted, 2GB default) |
|
|
||||||
|
|
||||||
**`config.toml` Example:**
|
### `~/.config/grel/config.toml`
|
||||||
```toml
|
```toml
|
||||||
[general]
|
[general]
|
||||||
version = 1
|
version = 1
|
||||||
@@ -149,106 +140,118 @@ max_concurrent = 4
|
|||||||
proxy = "http://127.0.0.1:7890"
|
proxy = "http://127.0.0.1:7890"
|
||||||
|
|
||||||
[assets]
|
[assets]
|
||||||
default_selection_policy = "largest" # first | largest | preferred_format
|
ignore_formats = ["*.deb", "*.rpm", "*.msi", "*.dmg", "*.pkg", "*.AppImage"]
|
||||||
prefer_formats = ["*.tar.gz", "*.zip", "*.exe"]
|
prefer_formats = ["*.tar.gz", "*.tar.xz", "*.zip", "*.exe"]
|
||||||
ignore_patterns = ["*source*", "*.deb", "*.rpm"]
|
arch_priority = ["x86_64", "aarch64", "x86", "armv7"]
|
||||||
|
fallback_to_32bit = true
|
||||||
|
prefer_musl = false
|
||||||
|
|
||||||
|
[migrations]
|
||||||
|
"legacy/old-tool" = "new-org/old-tool"
|
||||||
|
|
||||||
[upgrade]
|
[upgrade]
|
||||||
check_interval_hours = 6
|
check_interval_hours = 6
|
||||||
max_parallel_checks = 10
|
max_parallel_checks = 10
|
||||||
```
|
```
|
||||||
|
|
||||||
**Migrations:**
|
### `state.sqlite` Schema (`installed` table)
|
||||||
- Config: `version` field in TOML. On load, `grel-config` runs migration functions (rename keys, add defaults, warn on breaking changes).
|
```sql
|
||||||
- SQLite: `sqlx migrate` with timestamped files. Applied automatically on startup. Zero-downtime schema upgrades.
|
CREATE TABLE installed (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
forge TEXT NOT NULL, -- 'github', 'gitlab', or self-hosted base URL
|
||||||
|
owner TEXT NOT NULL,
|
||||||
|
repo TEXT NOT NULL,
|
||||||
|
version TEXT NOT NULL, -- Strict semver
|
||||||
|
asset_filename TEXT NOT NULL, -- Exact remote filename used
|
||||||
|
checksum TEXT,
|
||||||
|
install_path TEXT NOT NULL,
|
||||||
|
status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active', 'orphaned', 'migrated')),
|
||||||
|
orphaned_at INTEGER, -- Unix timestamp when marked unreachable
|
||||||
|
last_checked INTEGER, -- For smart polling intervals
|
||||||
|
installed_at INTEGER DEFAULT (strftime('%s', 'now'))
|
||||||
|
);
|
||||||
|
CREATE UNIQUE INDEX idx_pkg_unique ON installed(forge, owner, repo);
|
||||||
|
CREATE INDEX idx_status ON installed(status);
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7. Asset Resolution & Selection Pipeline
|
## 7. `-Syu` Upgrade Logic & Edge Cases
|
||||||
Releases contain multiple formats. `grel` uses **deterministic tiered matching** + **interactive fallback**.
|
### 7.1 Filename Change Detection
|
||||||
|
Authors often rename assets (`x86_64` → `amd64`, `.tar.gz` → `.tar.xz`).
|
||||||
|
1. Load stored `asset_filename` from DB
|
||||||
|
2. Resolve new release → filter → sort → get `new_best`
|
||||||
|
3. If `new_best.filename != stored.filename`:
|
||||||
|
```
|
||||||
|
⚠️ Remote asset renamed: old-name.tar.gz → new-name.tar.xz
|
||||||
|
ℹ️ Proceeding with update...
|
||||||
|
```
|
||||||
|
4. Download → verify → extract → **update DB record** with new filename. Proceeds safely.
|
||||||
|
|
||||||
### Tiered Matching (Order Matters)
|
### 7.2 Repository Rename / 404 Handling
|
||||||
1. **Exact:** `{repo}-{version}-{os}-{arch}.{fmt}`
|
1. Provider fetch → `404` or unreachable
|
||||||
2. **OS+Arch:** `*{os}*{arch}*.{fmt}`
|
2. Mark `status = 'orphaned'`, set `orphaned_at = now()`
|
||||||
3. **OS Only:** `*{os}*.{fmt}`
|
3. Skip in future `-Syu` runs
|
||||||
4. **Universal Archive:** `*.{tar.gz,tar.xz,zip}`
|
4. Warn user:
|
||||||
5. **Universal Binary:** `*.{exe,elf,dll,dylib}`
|
```
|
||||||
|
⚠️ Package unreachable: foo/bar (HTTP 404)
|
||||||
|
ℹ️ Skipped. Run `grel migrate foo/bar new/path` or `grel remove foo/bar`
|
||||||
|
```
|
||||||
|
5. **No auto-migration.** Explicit user action required.
|
||||||
|
|
||||||
### Checksum & Signature Auto-Pairing
|
### 7.3 Orphaned Visibility
|
||||||
If main asset is `foo-1.0.0-linux-amd64.tar.gz`, `grel` searches for:
|
|
||||||
- `foo-1.0.0-linux-amd64.tar.gz.sha256`
|
|
||||||
- `sha256sums.txt` (parses line matching filename)
|
|
||||||
Automatically downloads & verifies before extraction.
|
|
||||||
|
|
||||||
### Interactive Selection (Pacman/Artix Style)
|
|
||||||
- Uses `dialoguer::Select` with numbered list
|
|
||||||
- Default marker: `(default)` shown next to highest-priority match
|
|
||||||
- `--noconfirm` / `-y` skips prompt, uses default
|
|
||||||
- **TTY Detection:** If not interactive (CI/pipes), auto-selects default + prints `ℹ️ Non-interactive mode. Using default: <filename>` to stderr
|
|
||||||
- Timeout fallback (configurable): auto-selects default if no input within `prompt_timeout_secs`
|
|
||||||
|
|
||||||
**CLI Overrides:**
|
|
||||||
```bash
|
```bash
|
||||||
grel sync foo/bar --asset custom.exe
|
$ grel list
|
||||||
grel sync foo/bar --platform linux/aarch64
|
:: Installed packages (3 active, 1 orphaned)
|
||||||
grel sync foo/bar --list-assets # Dry-run table
|
🟢 bar/fuzz 1.2.3 linux/x86_64 tar.gz
|
||||||
|
🟢 foo/tool 0.9.1 windows/amd64 exe
|
||||||
|
🟡 legacy/old-proj 2.0.0 linux/x86_64 tar.gz [orphaned since 2024-05-01]
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 8. Networking, Proxy & DNS/IP Caching
|
## 8. Networking, Proxy & DNS/IP Caching
|
||||||
### Proxy Support
|
- **Proxy Priority:** `--proxy` > `GREL_PROXY` env > `http_proxy`/`all_proxy` > `config.toml`
|
||||||
- Native via `reqwest::Proxy::all()`, `http()`, `https()`
|
- **DNS/IP Cache:**
|
||||||
- Priority: `--proxy` > `GREL_PROXY` > `http_proxy`/`all_proxy` > `config.toml`
|
|
||||||
- Supports `http://user:pass@host`, `socks5://...`, `socks5h://...`
|
|
||||||
|
|
||||||
### DNS/IP Caching
|
|
||||||
⚠️ **No hardcoded IPs.** CDNs rotate frequently. Instead:
|
|
||||||
1. Resolve `A/AAAA` via `hickory-resolver`
|
1. Resolve `A/AAAA` via `hickory-resolver`
|
||||||
2. Parallel `TcpStream::connect` probes to all IPs on `443`
|
2. Parallel `TcpStream::connect` probes on `443`
|
||||||
3. Store fastest IP + RTT in SQLite with TTL (default `300s`)
|
3. Store fastest IP + RTT in SQLite with `300s` TTL
|
||||||
4. Use `reqwest::ClientBuilder::resolve(host, ip)` for routing
|
4. `reqwest::ClientBuilder::resolve(host, ip)` forces routing
|
||||||
5. Background refresh thread evicts stale entries, pre-warms hot domains
|
- **Background Refresh:** Idle task pre-warms hot domains, evicts stale entries
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 9. Rate Limiting & `-Syu` Optimization
|
## 9. Rate Limiting & ETag Optimization
|
||||||
GitHub allows 60 req/hr unauthenticated, 5000/hr authenticated. `grel` handles this gracefully:
|
GitHub: 60/hr unauth, 5000/hr auth. `grel` handles gracefully:
|
||||||
|
1. **ETag Caching:** `If-None-Match` → `304 Not Modified` **free**. Cached in DB.
|
||||||
|
2. **Smart Polling:** Only checks packages where `last_checked < now() - check_interval_hours`
|
||||||
|
3. **Rate Limit Parsing:** Reads `X-RateLimit-Remaining`/`Reset`. If `< 50` → sleep/queue.
|
||||||
|
4. **Auth Prompt:** First run suggests `GREL_GITHUB_TOKEN` for 5000/hr.
|
||||||
|
5. **`--no-api` Fallback:** Skips remote checks, uses local timestamps only.
|
||||||
|
|
||||||
1. **ETag Caching:** `If-None-Match` → `304 Not Modified` **does not count** against rate limits. Cached ETags stored in `state.sqlite`.
|
---
|
||||||
2. **Smart Polling:** Only recheck packages where `last_checked < now() - check_interval_hours`.
|
|
||||||
3. **Rate Limit Awareness:** Parse `X-RateLimit-Remaining` & `X-RateLimit-Reset`. If `< 50`, sleep until reset or queue remaining checks.
|
|
||||||
4. **Auth Token:** Strongly recommended. `grel` prompts on first run: `🔑 Set GREL_GITHUB_TOKEN for 5000 req/hr`.
|
|
||||||
5. **`--no-api` Fallback:** Skips remote checks, uses local manifest + cache timestamps.
|
|
||||||
6. **Parallel Batch Checks:** `max_parallel_checks` limits concurrent API calls to avoid hitting burst limits.
|
|
||||||
|
|
||||||
**`-Syu` Flow:**
|
## 10. PATH Integration & Post-Install
|
||||||
|
- Installs to `~/.local/share/grel/bin/` (Linux) / `%LOCALAPPDATA%\grel\bin\` (Windows)
|
||||||
|
- `grel path add` prints exact shell/registry snippets:
|
||||||
|
```bash
|
||||||
|
export PATH="$HOME/.local/share/grel/bin:$PATH"
|
||||||
```
|
```
|
||||||
Load state → Filter stale → Batch API (ETag) → Parse 200/304 → Build diff → Prompt → Download → Verify → Update
|
```powershell
|
||||||
|
[Environment]::SetEnvironmentVariable("Path", "$env:LOCALAPPDATA\grel\bin;$env:Path", "User")
|
||||||
```
|
```
|
||||||
|
- First run: `💡 Run: grel path add to enable global command access`
|
||||||
|
- XDG-compliant, no `sudo`, no system conflicts.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 10. Cross-Platform Implementation Notes
|
## 11. Security & Reliability
|
||||||
| Aspect | Linux | Windows | Implementation |
|
- ✅ TLS: `rustls` only (no native OpenSSL)
|
||||||
|--------|-------|---------|----------------|
|
- ✅ Checksums: `sha256` verification before extraction
|
||||||
| Paths | `~/.local/share/grel` | `%LOCALAPPDATA%\grel` | `directories::ProjectDirs` |
|
- ✅ Archive safety: Reject `..`, absolute paths, external symlinks
|
||||||
| Archives | `tar` + `zstd`/`xz`/`gz` | `zip` + `tar` | Feature-gated extraction |
|
- ✅ Atomic installs: Temp dir → verify → `rename` → DB update
|
||||||
| Exec Perms | `0o755` | `FILE_ATTRIBUTE_ARCHIVE` | `cfg(unix)` vs `cfg(windows)` |
|
- ✅ No auto-exec: Binaries installed but not run unless invoked
|
||||||
| Path Safety | Reject `..`, absolute paths | Same | `sanitize-filename` + strict validation |
|
- ✅ Graceful degradation: Network errors → retry/backoff, rate limits → sleep/queue, missing assets → clear error + suggestions
|
||||||
| Executable Suffix | None | `.exe` | Auto-append if `cfg(windows)` & missing |
|
|
||||||
| Shell Integration | `.bashrc`, `.zshrc`, fish | `$PROFILE`, Registry | `grel path add` |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11. Security & Reliability Guarantees
|
|
||||||
- ✅ **TLS:** `rustls` only (no native OpenSSL)
|
|
||||||
- ✅ **Checksums:** Verify `sha256` before extraction
|
|
||||||
- ✅ **Archive Safety:** Reject `..`, absolute paths, symlinks to outside dest
|
|
||||||
- ✅ **Atomic Installs:** Download to temp dir → verify → `rename` → register in DB
|
|
||||||
- ✅ **No Auto-Execute:** Binaries installed but not run unless invoked by user
|
|
||||||
- ✅ **Token Security:** Loaded from env/config, never logged or serialized to stdout
|
|
||||||
- ✅ **Graceful Degradation:** Network errors → retry with backoff, rate limits → sleep/queue, missing assets → clear error + suggestions
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -259,11 +262,10 @@ tokio = { version = "1", features = ["full"] }
|
|||||||
futures = "0.3"
|
futures = "0.3"
|
||||||
async-trait = "0.1"
|
async-trait = "0.1"
|
||||||
|
|
||||||
# CLI & UX
|
# CLI & UX (Pure CLI, no TUI)
|
||||||
clap = { version = "4", features = ["derive", "wrap_help", "string"] }
|
clap = { version = "4", features = ["derive", "wrap_help", "string"] }
|
||||||
indicatif = { version = "0.17", features = ["tokio"] }
|
indicatif = { version = "0.17", features = ["tokio"] }
|
||||||
tracing-indicatif = "0.3"
|
tracing-indicatif = "0.3"
|
||||||
dialoguer = "0.11"
|
|
||||||
tabled = "0.16"
|
tabled = "0.16"
|
||||||
owo-colors = "4"
|
owo-colors = "4"
|
||||||
|
|
||||||
@@ -288,23 +290,23 @@ figment = { version = "0.10", features = ["toml", "env"] }
|
|||||||
semver = "1"
|
semver = "1"
|
||||||
url = "2"
|
url = "2"
|
||||||
chrono = { version = "0.4", features = ["serde"] }
|
chrono = { version = "0.4", features = ["serde"] }
|
||||||
|
regex = "1"
|
||||||
serde = { version = "1", features = ["derive"] }
|
serde = { version = "1", features = ["derive"] }
|
||||||
anyhow = "1"
|
anyhow = "1"
|
||||||
thiserror = "2"
|
thiserror = "2"
|
||||||
tracing = "0.1"
|
tracing = "0.1"
|
||||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
||||||
```
|
```
|
||||||
*Note: Rust 1.70+ required for `std::io::IsTerminal` (replaces `atty`).*
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 13. Testing & CI Strategy
|
## 13. Testing & CI Strategy
|
||||||
- **Unit:** Parsing, version comparison, cache TTL math, asset matching tiers
|
- **Unit:** Token extraction from 20+ messy filenames, filter/sort determinism, config parsing, migration logic
|
||||||
- **Mocked Providers:** `wiremock` for API responses (200, 304, 403, 429, 500)
|
- **Mocked Providers:** `wiremock` for 200/304/403/429/500, rate limit header injection
|
||||||
- **Integration:** Download test fixtures → extract → verify structure/checksums
|
- **Integration:** Download fixtures → extract → verify checksums & paths, `grel migrate` DB state changes
|
||||||
|
- **Non-Interactive:** `echo "" | grel sync foo/bar` must not hang, must log to stderr
|
||||||
- **Cross-Platform CI:** `ubuntu-latest`, `windows-latest` (GitHub Actions)
|
- **Cross-Platform CI:** `ubuntu-latest`, `windows-latest` (GitHub Actions)
|
||||||
- **Performance:** `criterion` for DNS resolution + parallel download throughput
|
- **Performance:** `criterion` for DNS/IP cache + parallel download throughput
|
||||||
- **Fuzzing:** `cargo-fuzz` on `PackageRef` parser + archive path validation
|
- **Fuzzing:** `cargo-fuzz` on `AssetTokens::from_filename()` + archive path validation
|
||||||
- **Non-Interactive Tests:** `echo "" | grel sync foo/bar` must not hang
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
Reference in New Issue
Block a user