# Simulation Study Functions

## Overview

This package provides a comprehensive framework for comparing regression methods on image-based regression problems. It includes four main components:

1. **`sim_reg_comparison.m`** - Core comparison function for a single simulation run
2. **`run_simulation_study.m`** - Executes comprehensive simulations across multiple scenarios
3. **`analyze_simulation_results.m`** - Loads cached results and generates tables/plots
4. **`sim_reg_one_run.m`** - Quick demo showing different estimation methods, coefficient images, and predictions
5. **`sim_reg_full_run.m`** - Complete workflow script that runs simulations and analysis

---

## Quick Start

### Option 1: Quick Demo (Single Run)
```matlab
% Run a single simulation to see all methods
sim_reg_one_run;  % Shows coefficient images and prediction plots
```

### Option 2: Full Simulation Study
```matlab
% Run the complete analysis workflow
sim_reg_full_run;  % Runs simulation and generates all results
```

### Option 3: Custom Analysis
```matlab
% Add current directory to path
addpath(genpath(pwd));

% Step 1: Run simulations with custom repetitions
run_simulation_study(50);  % 50 repetitions (full study)
% run_simulation_study(5);   % Quick test with 5 reps

% Step 2: Analyze results
analyze_simulation_results();
```

---

## Simulation Parameters

### Scenarios (num_comp)
- **Scenario 1**: num_comp = 2 (2 active regions)
- **Scenario 2**: num_comp = 3 (3 active regions)
- **Scenario 3**: num_comp = 5 (5 active regions)

### SNR Levels
- 0, 0.25, 0.5, 0.75, 1.0

### Repetitions
- Configurable via `n_rep` parameter (default: 50)
- **Example**: 3 scenarios × 5 SNR levels × 50 reps = **750 simulations**

### Methods Compared
1. **RR** - Ridge Regression
2. **EN** - Elastic Net
3. **Lasso** - Lasso Regression
4. **FLR** - Fused Lasso Regression (baseline for S3IR)
5. **SREG** - Spatially Regularized Regression
6. **S3IR** - Spatially Structured Sparse Image Regression

---

## Function Details

### `sim_reg_comparison(num_comp, SNR, do_plot)`

**Purpose**: Run a single comparison of all methods

**Inputs**:
- `num_comp` - Number of regions/components (default: 3)
- `SNR` - Signal-to-noise ratio level (default: 1)
- `do_plot` - Generate plots (true/false, default: true)

**Outputs**:
- `results` - Structure containing all model results and metrics

**Example**:
```matlab
results = sim_reg_comparison(3, 1, true);  % 3 regions, SNR=1, with plots
results = sim_reg_comparison(5, 0.5, false);  % 5 regions, SNR=0.5, no plots
```

---

### `run_simulation_study(n_rep)`

**Purpose**: Execute comprehensive simulations across all scenarios and SNR levels

**Input**:
- `n_rep` (optional): Number of repetitions per combination (default: 50)

**What it does**:
- Loops through all scenario/SNR/repetition combinations
- Calls `sim_reg_comparison()` with plotting disabled
- Records REE and RMSPE for each method
- Saves results to `cached_results/simulation_results_TIMESTAMP.mat`

**Progress tracking**:
- Shows completion percentage
- Displays elapsed time and estimated remaining time
- Reports any errors during execution

**Usage examples**:
```matlab
run_simulation_study();      % Default: 50 reps → 750 total runs
run_simulation_study(100);   % Extended: 100 reps → 1500 total runs
run_simulation_study(5);     % Quick test: 5 reps → 75 total runs
```

**Output**: 
- Cached `.mat` file with timestamp in filename
- Results organized as: `all_results.scenario_X.SNR_Y.REE/RMSPE.MethodName`

---

### `analyze_simulation_results(results_file)`

**Purpose**: Load cached results and generate comprehensive outputs

**Input**:
- `results_file` (optional): Path to specific .mat file
- If empty or not provided, automatically uses most recent file in `cached_results/`

**What it generates**:

#### 1. Console Output Tables
For each scenario, prints tables with methods as rows and SNR levels as columns:
```
REE Mean(Std)
Method      SNR=0.00       SNR=0.25       SNR=0.50       SNR=0.75       SNR=1.00
-------------------------------------------------------------------------------
RR          0.12(0.02)     0.10(0.01)     0.09(0.01)     ...
EN          0.15(0.02)     0.11(0.01)     0.10(0.01)     ...
```

#### 2. Visualization Plots
- **Individual scenario figures**: One figure per scenario showing REE and RMSPE
- **Combined comparison figure**: All scenarios side-by-side
- **Plot elements**:
  - Dashed lines connecting median values
  - Asterisks (*) marking medians
  - Crosses (×) marking 25th/75th percentiles

#### 3. CSV Export Files
Saved to `simulation_tables/` folder:
- `REE_scenario1_numcomp2.csv`
- `RMSPE_scenario1_numcomp2.csv`
- `REE_scenario2_numcomp3.csv`
- `RMSPE_scenario2_numcomp3.csv`
- `REE_scenario3_numcomp5.csv`
- `RMSPE_scenario3_numcomp5.csv`

Each CSV format:
- Rows: Methods (RR, EN, Lasso, FLR, SREG, S3IR)
- Columns: SNR levels (0.00, 0.25, 0.50, 0.75, 1.00)
- Cell values: mean(std) format

---

### `sim_reg_one_run.m`

**Purpose**: Quick demonstration of all methods on a single dataset

**What it does**:
- Generates one synthetic dataset
- Fits all six regression methods
- Displays coefficient image comparisons (Figure 1)
- Shows prediction scatter plots (Figure 2)
- Presents performance metric bar charts (Figure 3)
- Prints summary table to console

**Usage**:
```matlab
sim_reg_one_run;  % Run the demo
```

**Output**:
- 3 figures visualizing results
- Console output with performance metrics

---

### `sim_reg_full_run.m`

**Purpose**: Complete workflow combining simulation and analysis

**What it does**:
1. Calls `run_simulation_study(n_rep)` to execute simulations
2. Calls `analyze_simulation_results()` to generate outputs
3. Automates the entire process from data generation to final plots

**Usage**:
```matlab
sim_reg_full_run;  % Runs complete analysis with default settings
```

**Note**: Edit the file to customize `n_rep` before running

---

## Output Interpretation

### Performance Metrics

**REE (Relative Estimation Error)**:
- Measures how well estimated coefficients match true coefficients
- Formula: `||β_estimated - β_true||² / ||β_true||²`
- **Lower is better** (0 = perfect recovery)

**RMSPE (Root Mean Squared Prediction Error)**:
- Measures prediction accuracy on test data
- Formula: `sqrt(mean((y_predicted - y_true)²))`
- **Lower is better** (0 = perfect prediction)

**Correlation**:
- Correlation between predicted and true test responses
- **Higher is better** (1 = perfect correlation)

### Table Format
```
Method      SNR=0.00       SNR=0.25       SNR=0.50       SNR=0.75       SNR=1.00
-------------------------------------------------------------------------------
RR          0.12(0.02)     0.10(0.01)     0.09(0.01)     0.08(0.01)     0.07(0.01)
EN          0.15(0.02)     0.11(0.01)     0.10(0.01)     0.09(0.01)     0.08(0.01)
Lasso       0.16(0.02)     0.12(0.02)     0.11(0.01)     0.10(0.01)     0.09(0.01)
```
- **Rows**: Methods
- **Columns**: SNR levels
- **Values**: mean(std) across repetitions
- All values shown to 2 decimal places

### Plot Interpretation
- **X-axis**: SNR level (higher = less noise)
- **Y-axis**: Performance metric (REE or RMSPE)
- **Lines**: Dashed lines connecting medians
- **Asterisks (*)**: Median values (50th percentile)
- **Crosses (×)**: 25th and 75th percentiles (interquartile range)
- **Lower curves are better** for both metrics

### Color Scheme
- **RR (Ridge)** - Green `[0.0, 0.8, 0.0]`
- **EN (Elastic Net)** - Pink `[1.0, 0.4, 0.7]`
- **Lasso** - Cyan `[0.0, 0.8, 0.8]`
- **FLR** - Blue `[0.0, 0.0, 1.0]`
- **SREG** - Black `[0.0, 0.0, 0.0]`
- **S3IR** - Red `[1.0, 0.0, 0.0]`

---

## Folder Structure

```
S3IR_project/
├── sim_reg_comparison.m          # Core comparison function
├── run_simulation_study.m        # Comprehensive simulation runner
├── analyze_simulation_results.m  # Results analyzer and visualizer
├── sim_reg_one_run.m             # Quick demo (single run)
├── sim_reg_full_run.m            # Complete workflow script
├── SIMULATION_STUDY_README.md    # This documentation
│
├── SMAC/                         # Required: SREG algorithm package
├── S3IR/                         # Required: S3IR algorithm package
│
├── cached_results/               # Auto-created: Simulation output
│   └── simulation_results_20251006_143022.mat
│
└── simulation_tables/            # Auto-created: CSV exports
    ├── REE_scenario1_numcomp2.csv
    ├── RMSPE_scenario1_numcomp2.csv
    ├── REE_scenario2_numcomp3.csv
    ├── RMSPE_scenario2_numcomp3.csv
    ├── REE_scenario3_numcomp5.csv
    └── RMSPE_scenario3_numcomp5.csv
```

---

## Usage Tips

### Getting Started
1. **First time**: Run `sim_reg_one_run` to see a quick demo
2. **Quick test**: Run `run_simulation_study(5)` to test the full workflow (~50 min)
3. **Full study**: Run `run_simulation_study(50)` or use `sim_reg_full_run` (~8-10 hours)
4. **Re-analyze**: Run `analyze_simulation_results()` to regenerate plots/tables

### Performance & Runtime
Approximate runtimes on a typical workstation:

| Repetitions | Total Runs | Estimated Time | Use Case |
|-------------|------------|----------------|----------|
| 5           | 75         | ~50 min        | Quick test |
| 10          | 150        | ~1.5-2 hours   | Fast preliminary |
| 20          | 300        | ~3-4 hours     | Preliminary study |
| 50          | 750        | ~8-10 hours    | Full study (recommended) |
| 100         | 1500       | ~16-20 hours   | Extended study |

**Recommendation**: Start with 5 reps (~50 min) to verify everything works, then run overnight with 50-100 reps for publication-quality results.

### Re-running Analysis Only
If you want to regenerate plots/tables without re-running simulations:
```matlab
analyze_simulation_results();  % Uses most recent cached results
```

### Using Specific Cached Results
```matlab
analyze_simulation_results('cached_results/simulation_results_20251006_143022.mat');
```

### Customization Options

**Change scenarios**:
Edit `run_simulation_study.m`:
```matlab
num_comp_values = [2, 3, 5];  % Modify this array
```

**Change SNR levels**:
Edit `run_simulation_study.m`:
```matlab
SNR_values = 0:0.25:1;  % Modify this array
```

**Change colors**:
Edit `analyze_simulation_results.m`:
```matlab
colors = [
    0.0 0.8 0.0;     % RR - change to your preferred RGB
    1.0 0.4 0.7;     % EN
    ...
];
```

**Change image size**:
Edit `sim_reg_comparison.m`:
```matlab
D1 = 48;  % Image height
D2 = 48;  % Image width
```

---

## Expected Results

### General Trends
- **REE decreases** as SNR increases (less noise → better coefficient estimation)
- **RMSPE decreases** as SNR increases (better estimation → better prediction)
- **Methods diverge** at low SNR (difficult conditions reveal differences)
- **Methods converge** at high SNR (easy conditions → similar performance)

### Method Comparisons
- **S3IR** typically best at exploiting spatial structure
- **SREG** good at spatial smoothness
- **Lasso/EN** provide sparsity
- **RR (Ridge)** provides stable baseline
- **FLR** shows benefit of spatial constraints

### Interpretation Guidelines
- **Best method overall**: Consistently lowest REE and RMSPE across SNR levels
- **Most robust**: Smallest std (shown in parentheses) across repetitions
- **Best at low SNR**: Lowest values when SNR ≤ 0.5
- **Best at high SNR**: Lowest values when SNR ≥ 0.75

---