# Hybrid Diffusion for ICS Traffic (HAI 21.03) — Project Report
# 工业控制系统流量混合扩散生成（HAI 21.03）— 项目报告

## 1. Project Goal / 项目目标
Build a **hybrid diffusion-based generator** for ICS traffic features, focusing on **mixed continuous + discrete** feature sequences. The output is **feature-level sequences**, not raw packets. The generator should preserve:
- **Distributional fidelity** (continuous ranges + discrete frequencies)
- **Temporal consistency** (time correlation and sequence structure)
- **Field/logic consistency** for discrete protocol-like columns

构建一个用于 ICS 流量特征的**混合扩散生成模型**，处理**连续+离散混合特征序列**。输出为**特征级序列**而非原始报文。生成结果需要保持：
- **分布一致性**（连续值范围 + 离散频率）
- **时序一致性**（时间相关性与序列结构）
- **字段/逻辑一致性**（离散字段语义）

---

## 2. Data and Scope / 数据与范围
**Dataset used in current implementation:** HAI 21.03 (CSV feature traces).

**当前实现使用数据集：** HAI 21.03（CSV 特征序列）。

**Data path (default in config):**
- `dataset/hai/hai-21.03/train*.csv.gz`

**特征拆分（固定 schema）：** `example/feature_split.json`
- Continuous features: sensor/process values
- Discrete features: binary/low-cardinality status/flag fields
- `time` is excluded from modeling

---

## 3. End-to-End Pipeline / 端到端流程
One command pipeline:
```
python example/run_all.py --device cuda
```

Pipeline stages:
1) **Prepare data** (`example/prepare_data.py`)
2) **Train model** (`example/train.py`)
3) **Generate samples** (`example/export_samples.py`)
4) **Evaluate** (`example/evaluate_generated.py`)

一键流程对应：数据准备 → 训练 → 采样导出 → 评估。

---

## 4. Technical Architecture / 技术架构

### 4.1 Hybrid Diffusion Model (Core) / 混合扩散模型（核心）
Defined in `example/hybrid_diffusion.py`.

**Inputs:**
- Continuous projection
- Discrete embeddings
- Time embedding (sinusoidal)
- Positional embedding (sequence index)
- Optional condition embedding (`file_id`)

**Backbone:**
- GRU (sequence modeling)
- Post LayerNorm + residual MLP

**Outputs:**
- Continuous head: predicts target (`eps` or `x0`)
- Discrete heads: logits per discrete column

**连续分支：** Gaussian diffusion
**离散分支：** Mask diffusion

---

### 4.2 Temporal Backbone (GRU) / 共享时序骨干（GRU）
The GRU is the **shared temporal backbone** that fuses continuous + discrete signals into a unified sequence representation, enabling joint modeling of temporal dynamics and cross-feature dependencies.

GRU 是模型的**共享时序核心**，把连续/离散特征统一建模在同一时间结构中。

---

## 5. Diffusion Formulations / 扩散形式

### 5.1 Continuous Diffusion / 连续扩散
Forward process:
```
x_t = sqrt(a_bar_t) * x_0 + sqrt(1 - a_bar_t) * eps
```
Targets supported:
- **eps prediction** (default)
- **x0 prediction** (direct reconstruction)

Current config:
```
"cont_target": "x0"
```

### 5.2 Discrete Diffusion / 离散扩散
Mask diffusion with cosine schedule:
```
p(t) = 0.5 * (1 - cos(pi * t / T))
```
Mask-only cross-entropy is computed on masked positions.

---

## 6. Loss Design / 损失设计
Total loss:
```
L = λ * L_cont + (1 − λ) * L_disc
```

### 6.1 Continuous Loss / 连续损失
- `eps` target: MSE(eps_pred, eps)
- `x0` target: MSE(x0_pred, x0)
- Optional inverse-variance weighting: `cont_loss_weighting = "inv_std"`

### 6.2 Discrete Loss / 离散损失
Cross-entropy on masked positions only.

---

## 7. Data Processing / 数据处理
Defined in `example/data_utils.py` + `example/prepare_data.py`.

Key steps:
- Streaming mean/std/min/max + int-like detection
- Optional **log1p transform** for heavy-tailed continuous columns
- Discrete vocab + most frequent token
- Windowed batching with **shuffle buffer**

---

## 8. Sampling & Export / 采样与导出
Defined in:
- `example/sample.py`
- `example/export_samples.py`

Export process:
- Reverse diffusion sampling
- De-normalize continuous values
- Clamp to observed min/max
- Restore discrete tokens from vocab
- Write to CSV

---

## 9. Evaluation / 评估指标
Defined in `example/evaluate_generated.py`.

Metrics (with reference):
- **KS statistic** (continuous distribution)
- **Quantile diffs** (q05/q25/q50/q75/q95)
- **Lag‑1 correlation diff** (temporal structure)
- **Discrete JSD** over vocab frequency
- **Invalid token counts**

---

## 10. Automation / 自动化
`example/run_all.py` runs all stages with config-driven paths.

---

## 11. Key Engineering Decisions / 关键工程决策
- Mixed-type diffusion: continuous + discrete split
- Shared temporal backbone (GRU) to align sequence structure
- Positional + time embeddings for stability
- Optional inverse-variance weighting for continuous loss
- Log1p transforms for heavy-tailed signals

---

## 12. Code Map (Key Files) / 代码索引
- Core model: `example/hybrid_diffusion.py`
- Training: `example/train.py`
- Data prep: `example/prepare_data.py`
- Data utilities: `example/data_utils.py`
- Sampling: `example/sample.py`
- Export: `example/export_samples.py`
- Evaluation: `example/evaluate_generated.py`
- Pipeline: `example/run_all.py`
- Config: `example/config.json`

---

## 13. Known Issues / Current Limitations / 已知问题
- KS sometimes remains high → continuous distribution mismatch
- Lag‑1 may fluctuate → distribution vs temporal trade-off
- Continuous loss may dominate → needs careful weighting

---

## 14. Suggested Next Steps / 下一步建议
- Add **SNR-weighted loss** for stable diffusion training
- Explore **v‑prediction** for continuous branch
- Consider **two-stage training** (temporal first, distribution second)
- Strengthen discrete diffusion (e.g., D3PM-style transitions)

---

## 15. Summary / 总结
This project implements a **hybrid diffusion model** for ICS feature sequences with a GRU backbone, handling continuous and discrete features separately while sharing temporal structure. The pipeline covers data prep, training, sampling, export, and evaluation. The main research challenge remains in balancing **distributional fidelity (KS)** and **temporal consistency (lag‑1)**.

本项目实现了基于 GRU 的混合扩散模型，连续/离散分支分开建模但共享时序结构，具备完整的训练与评估流程。主要挑战是**分布对齐（KS）与时序一致性（lag‑1）之间的平衡**。

---

## 16. Latest Evaluation Snapshot / 最新评估快照
Computed averages from the latest `eval.json`:
- **avg_ks**: 0.5208903596698115
- **avg_jsd**: 0.010592151023360712
- **avg_lag1_diff**: 0.8265139723919303

最新评估均值（来自 `eval.json`）：
- **avg_ks**：0.5208903596698115
- **avg_jsd**：0.010592151023360712
- **avg_lag1_diff**：0.8265139723919303