improved project saving capabilities

changelog.md

@@ -1,6 +1,6 @@
 # Verison 1.6.0
 
-- This is potentially a save-changing release due to adding more data into the save file and renaming existing data. Please update your project files to ensure compatibility
+- This is potentially a save-changing release due to adding more data into the save file, renaming existing data, and changing what part of the code saves data. Please update your project files to ensure compatibility
 - It is still possible to load older saves by enabling 'Incompatible Save Bypass' from the Preferences menu, but your mileage may vary
 - Optimized calculations being performed when calculating the heart rate to speed up step 5 by up to ~35% on a per-file basis
 - Optimized calculations being performed when running the General Linear Model to speed up step 5 by ~35% on a per-file basis
@@ -13,6 +13,17 @@
 - Added a new metadata value of HAND to allow for grouping participants based on their handiness
 - Updated the layout of the participant metadata area to better accommodate current and future metadata values
 - Updated the message displayed when "Why are these useful?" is clicked to better represent what the values are used for and to accomidate the new metadata values
+- Fixed some popup windows not properly displaying the application name
+- Changed the application title to now show what project you are working on. If you are not working on a project, it will display 'Untitled'
+- Tracking of current save states is now present and an asterisks will now appear in the title bar when a project has not been saved
+- Currently tracking is only supported on loaded files, parameters on the right side of the screen, and when processing has completed. Per-file metadata is not currently implemented
+- Clicking close on the Main Window will now properly close all subwindows when closing the application, and closing the application with unsaved changes will now prompt to save a project or discard changes.  Fixes [Issue 87](https://git.research.dezeeuw.ca/tyler/flares/issues/87)
+- The Save button under the File menu no longer asks for a destination if a saved project was already loaded. Fixes [Issue 72](https://git.research.dezeeuw.ca/tyler/flares/issues/72)
+- The popup that appeared when a project was saved has now been moved to a message displayed from the status bar
+- Fixed an issue where when loading a save file some list dropdowns could go blue and bold even if the value was default
+- Moved all loading of files, loading of folders, loading of projects, and saving of projects to a new file for easier future development
+- Changed the working of some messages on the status bar to better reflect what has occurred
+- Fixed an issue where the Analysis button would not be clickable if the user had previously pushed Clear. Fixes [Issue 83](https://git.research.dezeeuw.ca/tyler/flares/issues/83)
 
 
 # Version 1.5.2
@@ -62,7 +73,7 @@
 - Fixed participant metadata remaining in the background when the participant was removed. Fixes [Issue 82](https://git.research.dezeeuw.ca/tyler/flares/issues/82)
 - Fixed processing remaining active hidden in the background when the "Clear" button was pushed. Fixes [Issue 81](https://git.research.dezeeuw.ca/tyler/flares/issues/81)
 - Now when "Clear" is pushed while data is processing, a popup will appear ensuring that the user understands that pushing "Clear" will stop processing
-- Reset to Default Configuration will now properly reset all of the parameters to their default values. [Issue 90](https://git.research.dezeeuw.ca/tyler/flares/issues/90)
+- Reset to Default Configuration will now properly reset all of the parameters to their default values. Fixes [Issue 90](https://git.research.dezeeuw.ca/tyler/flares/issues/90)
 - A confirmation popup will now display once the application has been reset to default in addition to the status bar message
 - Changed the improper display text of "Cross Validation" to now properly read as "Coefficient of Variation"
 - Changed the parameters "CV" and "CV_THRESHOLD" to now be "COEFF_VAR and COEFF_VAR_THRESHOLD"
@@ -81,7 +92,7 @@
 - Fixed a crucial bug where short channels were not being processed and filtered the same way as long channels before being used as regressors
 - Fixed a crucial bug where short channels were being presented to the design matrix as normal long channels
 - Fixed a crucial bug where long channels could be interpolated from short channels. Short channels are still potentially interpolated from long channels. See [this link](https://git.research.dezeeuw.ca/tyler/flares/issues/80) for more information regarding this issue.
-- Decreased unnecessary processing time when fOLDing channels by an order of magnitude
+- Decreased unnecessary processing time when fOLDing channels by an order of magnitude. Fixes [Issue 84](https://git.research.dezeeuw.ca/tyler/flares/issues/84)
 - Added a welcome message when the terminal is opened, resized the terminal, and added more commands
 
 

changelog_major.md

@@ -12,9 +12,9 @@
 - New parameters have been added to the right side of the screen! This allows for more flexibility and customizability when processing
 - A new Preference Menu option has been added: Show Advanced Parameters. This keeps some of the parameters hidden when not checked. Since this is a preference, it will be saved when reopening the application
 - Advanced parameters should only be changed if you know what you are doing, and will have a yellow warning symbol next to them to avoid potential confusion on what parameters are advanced
-- Optimized some of the calculations in Scalp Coupling Index to speed up Step 6 by ~25%
-- Removed duplicate/redundant calculations in Peak Spectral Power to speed up Step 8 by ~50%
-- Changed how the figures are generated when processing to speed up Step 28 by ~85%
+- Optimized some of the calculations in Scalp Coupling Index to speed up Step 6 by up to ~25% on a per-file basis
+- Removed duplicate/redundant calculations in Peak Spectral Power to speed up Step 8 by up to ~50% on a per-file basis
+- Changed how the figures are generated when processing to speed up Step 28 by up to ~85% on a per-file basis
 - Removed unused methods inside the processing file to slightly speed up application load time
 - Fixed an issue with the build script not properly updating the version string causing the application to falsely think that an update was always available
 - Fixed an issue where parameters that were dependent on SHORT_CHANNELS were not properly being updated
@@ -45,7 +45,7 @@
 - Fixed participant metadata remaining in the background when the participant was removed. Fixes [Issue 82](https://git.research.dezeeuw.ca/tyler/flares/issues/82)
 - Fixed processing remaining active hidden in the background when the "Clear" button was pushed. Fixes [Issue 81](https://git.research.dezeeuw.ca/tyler/flares/issues/81)
 - Now when "Clear" is pushed while data is processing, a popup will appear ensuring that the user understands that pushing "Clear" will stop processing
-- Reset to Default Configuration will now properly reset all of the parameters to their default values. [Issue 90](https://git.research.dezeeuw.ca/tyler/flares/issues/90)
+- Reset to Default Configuration will now properly reset all of the parameters to their default values. Fixes [Issue 90](https://git.research.dezeeuw.ca/tyler/flares/issues/90)
 - A confirmation popup will now display once the application has been reset to default in addition to the status bar message
 - Changed the improper display text of "Cross Validation" to now properly read as "Coefficient of Variation"
 - Changed the parameters "CV" and "CV_THRESHOLD" to now be "COEFF_VAR and COEFF_VAR_THRESHOLD"
@@ -64,7 +64,7 @@
 - Fixed a crucial bug where short channels were not being processed and filtered the same way as long channels before being used as regressors
 - Fixed a crucial bug where short channels were being presented to the design matrix as normal long channels
 - Fixed a crucial bug where long channels could be interpolated from short channels. Short channels are still potentially interpolated from long channels. See [this link](https://git.research.dezeeuw.ca/tyler/flares/issues/80) for more information regarding this issue.
-- Decreased unnecessary processing time when fOLDing channels by an order of magnitude
+- Decreased unnecessary processing time when fOLDing channels by an order of magnitude. Fixes [Issue 84](https://git.research.dezeeuw.ca/tyler/flares/issues/84)
 - Added a welcome message when the terminal is opened, resized the terminal, and added more commands
 
 

project_manager.py

@@ -0,0 +1,826 @@
+"""
+Filename: project_manager.py
+Description: Manager file for anything project related
+
+Author: Tyler de Zeeuw
+License: GPL-3.0
+"""
+
+# Built-in imports
+import os
+import sys
+import copy
+import pickle
+import concurrent
+from pathlib import Path, PurePosixPath
+
+# External library imports
+import pandas as pd
+
+from PySide6.QtWidgets import QMessageBox, QVBoxLayout, QFileDialog, QLabel, QDialog
+from PySide6.QtCore import QThread, Signal, Qt, QTimer
+from PySide6.QtGui import QAction
+
+from mne.io import read_raw_snirf
+from mne.preprocessing.nirs import source_detector_distances
+from mne_nirs.channels import get_short_channels  # type: ignore
+
+from src.shared.flaresbasewidget import ProgressBubble
+from src.shared.shareddata import APP_NAME, CURRENT_VERSION, PLATFORM_NAME, DATA_SCHEMA
+
+
+
+class SaveProjectThread(QThread):
+    finished_signal = Signal(str)
+    error_signal = Signal(str)
+
+    def __init__(self, filename, project_data):
+        super().__init__()
+        self.filename = filename
+        self.project_data = project_data
+
+    def run(self):
+        try:
+            with open(self.filename, "wb") as f:
+                pickle.dump(self.project_data, f)
+            self.finished_signal.emit(self.filename)
+        except Exception as e:
+            self.error_signal.emit(str(e))
+
+
+
+class SavingOverlay(QDialog):
+    def __init__(self, parent=None):
+        super().__init__(parent)
+        self.setWindowFlags(Qt.WindowType.Dialog | Qt.WindowType.FramelessWindowHint)
+        self.setModal(True)
+        self.setWindowModality(Qt.WindowModality.ApplicationModal)
+        self.setAttribute(Qt.WidgetAttribute.WA_TranslucentBackground)
+        
+        layout = QVBoxLayout()
+        layout.setAlignment(Qt.AlignmentFlag.AlignCenter)
+
+        label = QLabel("Saving Project…")
+        label.setStyleSheet("font-size: 18px; color: white; background-color: rgba(0,0,0,150); padding: 20px; border-radius: 10px;")
+        layout.addWidget(label)
+        self.setLayout(layout)
+        
+
+class ProjectManager:
+    """
+    Central manager for all I/O operations:
+    - File loading (individual files & folders)
+    - Project loading & saving (Save vs Save As)
+    - Relative / absolute path utilities
+    - State baseline synchronization (dirty tracking)
+    """
+
+    def __init__(self, app, file_cfg, cfg_path):
+        self.app = app
+        self.file_cfg = file_cfg
+        self.cfg_path = cfg_path
+
+    # =========================================================================
+    # Path Utilities
+    # =========================================================================
+    def get_safe_path(self, target_path, project_dir):
+        """Converts an absolute file path to a relative path relative to project_dir."""
+        try:
+            target = Path(target_path).resolve()
+            proj = Path(project_dir).resolve()
+            return str(PurePosixPath(target.relative_to(proj)))
+        except ValueError:
+            # Fall back to absolute path string if on a different drive/volume
+            return str(PurePosixPath(Path(target_path).resolve()))
+
+    # =========================================================================
+    # File & Folder Opening Dialogs
+    # =========================================================================
+    def open_file_dialog(self):
+        """Opens dialog to pick a single .snirf file."""
+        file_path, _ = QFileDialog.getOpenFileName(
+            self.app, "Open File", "", "SNIRF Files (*.snirf);;All Files (*)"
+        )
+        if file_path:
+            self._load_files_into_pipeline([os.path.normpath(file_path)])
+
+    def open_folder_dialog(self):
+        """Recursively finds all .snirf files in a selected directory."""
+        folder_path = QFileDialog.getExistingDirectory(self.app, "Select Folder", "")
+        if folder_path:
+            snirf_files = [os.path.normpath(str(f)) for f in Path(folder_path).rglob("*.snirf")]
+            self._load_files_into_pipeline(snirf_files)
+
+    def _load_files_into_pipeline(self, file_paths):
+        """Loads .snirf files into UI using chunked batches and background workers."""
+        app = self.app
+        if not file_paths:
+            return
+
+        # 1. Warm up the executor if needed
+        if not hasattr(app, "file_executor") or app.file_executor is None:
+            app.file_executor = concurrent.futures.ProcessPoolExecutor(max_workers=1)
+
+        # 2. Track this session to prevent ghost updates
+        if not hasattr(app, "loading_session_id"):
+            app.loading_session_id = 0
+        app.loading_session_id += 1
+        current_session = app.loading_session_id
+
+        # 3. Setup internal tracking if not exists
+        if not hasattr(app, "bubble_widgets"):
+            app.bubble_widgets = {}
+        if not hasattr(app, "selected_paths"):
+            app.selected_paths = []
+        if not hasattr(app, "metadata_cache"):
+            app.metadata_cache = {}
+
+        # Filter out files already in the UI to avoid duplicates
+        new_files = [p for p in file_paths if p not in app.selected_paths]
+        if not new_files:
+            return
+
+        # Update the pending count for the current load batch
+        if not hasattr(app, "pending_files_count"):
+            app.pending_files_count = 0
+        app.pending_files_count += len(new_files)
+
+        app.button1.setVisible(True)
+        app.statusBar().showMessage(f"Loading {len(new_files)} new file(s)...")
+
+        # Queue chunked widget creation
+        CHUNK_SIZE = 10
+
+        def process_chunk(file_queue):
+            chunk = file_queue[:CHUNK_SIZE]
+            remaining = file_queue[CHUNK_SIZE:]
+
+            for path in chunk:
+                app.selected_paths.append(path)
+                self.add_to_recent_files(path)
+
+                display_name = os.path.basename(path)
+                bubble = ProgressBubble(display_name, path)
+                bubble.setCursor(Qt.CursorShape.WaitCursor)
+                bubble.set_loading_state(True)
+
+                app.bubble_widgets[path] = bubble
+                app.bubble_layout.addWidget(bubble)
+
+                # Submit background task as each bubble is constructed
+                future = app.file_executor.submit(extract_metadata_worker, path)
+                future.add_done_callback(
+                    lambda f, p=path, s=current_session: app._on_metadata_ready(f, p, s)
+                )
+
+            app.files_are_dirty = True
+            app.is_saved = False
+
+            if hasattr(app, "check_if_app_is_dirty"):
+                app.check_if_app_is_dirty()
+            elif hasattr(app, "update_window_title"):
+                app.update_window_title()
+
+            # Schedule remaining files
+            if remaining:
+                QTimer.singleShot(0, lambda: process_chunk(remaining))
+
+        process_chunk(new_files)
+
+    def add_files_to_project(self, file_paths):
+        """Adds file paths to the application state, creating bubble UI items."""
+        app = self.app
+        normalized_paths = [os.path.normpath(p) for p in file_paths]
+
+        # Merge with existing selected paths avoiding duplicates
+        existing_paths = getattr(app, "selected_paths", [])
+        new_paths = [p for p in normalized_paths if p not in existing_paths]
+
+        if not new_paths:
+            return
+
+        app.selected_paths = existing_paths + new_paths
+
+        # Render file bubbles in UI if method exists
+        if hasattr(app, "show_files_as_bubbles_from_list"):
+            progress_states = getattr(app, "progress_states", {})
+            current_project = getattr(app, "current_project_path", "")
+            app.show_files_as_bubbles_from_list(
+                app.selected_paths, progress_states, current_project
+            )
+
+        # Record in recent files menu
+        for path in new_paths:
+            self.add_to_recent_files(path)
+
+        # Trigger dirty check
+        if hasattr(app, "check_if_app_is_dirty"):
+            app.check_if_app_is_dirty()
+
+    # =========================================================================
+    # Project Loading
+    # =========================================================================
+    def load_project_dialog(self):
+        """Prompts for a project file and loads it."""
+        app = self.app
+        filename, _ = QFileDialog.getOpenFileName(
+            app, "Load Project", "", "FLARE Project (*.flare)"
+        )
+        if filename:
+            self.load_project(filename)
+
+    def load_project(self, filename):
+        """Loads a .flare project file into the application."""
+        app = self.app
+        try:
+            with open(filename, "rb") as f:
+                data = pickle.load(f)
+
+            checks = [
+                ("version", "<=1.1.7"),
+                ("file_metadata", "<=1.2.2"),
+                ("file_parameters", "<=1.3.0"),
+                ("roi_channel_map_dict", "<=1.5.2"),
+            ]
+
+            for key, ver_str in checks:
+                if key not in data:
+                    msg = (
+                        f"This project was saved in an earlier version of {APP_NAME.upper()} ({ver_str}) "
+                        "and is potentially not compatible with this version. "
+                    )
+                    if getattr(app, "incompatible_save_bypass", False):
+                        QMessageBox.warning(
+                            app, f"Warning - {APP_NAME.upper()}", msg + "Attempting load."
+                        )
+                        break
+                    else:
+                        QMessageBox.critical(
+                            app,
+                            f"Error - {APP_NAME.upper()}",
+                            msg + "Enable bypass in Preferences to load.",
+                        )
+                        return
+
+            # Clear existing UI bubbles
+            if hasattr(app, "bubble_widgets"):
+                for bubble in list(app.bubble_widgets.values()):
+                    bubble.setParent(None)
+                    bubble.deleteLater()
+                app.bubble_widgets.clear()
+
+            app.selected_paths = []
+            app.current_project_path = filename
+
+            # Restore Data Schema
+            for item in DATA_SCHEMA:
+                setattr(app, item["key"], data.get(item["key"], {}))
+
+            project_dir = Path(filename).parent
+            saved_cache = data.get("file_metadata", {})
+            raw_params = data.get("file_parameters", {})
+            app.metadata_cache = {}
+            app.file_metadata = {}
+
+            for rel_path, meta_content in saved_cache.items():
+                abs_path = str((project_dir / Path(rel_path)).resolve())
+                app.metadata_cache[abs_path] = meta_content
+
+            file_list = [
+                str((project_dir / Path(rel_path)).resolve()) for rel_path in data["file_list"]
+            ]
+            raw_progress = data.get("progress_states", {})
+            progress_states = {
+                str((project_dir / Path(rel_path)).resolve()): step
+                for rel_path, step in raw_progress.items()
+            }
+
+            for rel_path in data["file_list"]:
+                abs_path = str((project_dir / Path(rel_path)).resolve())
+                if rel_path in raw_params:
+                    app.file_metadata[abs_path] = raw_params[rel_path]
+                elif hasattr(app, "config_dict") and abs_path in app.config_dict:
+                    old_cfg = app.config_dict[abs_path]
+                    app.file_metadata[abs_path] = {
+                        "AGE": str(old_cfg.get("AGE", "")),
+                        "SEX": str(old_cfg.get("SEX", "")),
+                        "HAND": str(old_cfg.get("HAND", "")),
+                        "GROUP": str(old_cfg.get("GROUP", "")),
+                    }
+                else:
+                    app.file_metadata[abs_path] = {
+                        "AGE": "",
+                        "SEX": "",
+                        "HAND": "",
+                        "GROUP": "",
+                    }
+
+            app.show_files_as_bubbles_from_list(file_list, progress_states, filename)
+
+            if "current_ui_params" in data:
+                app.restore_sections_from_config(data["current_ui_params"])
+            elif getattr(app, "config_dict", None):
+                first_file = next(iter(app.config_dict.keys()))
+                app.restore_sections_from_config(app.config_dict[first_file])
+
+            has_data = any(len(getattr(app, item["key"], {})) > 0 for item in DATA_SCHEMA)
+            if hasattr(app, "button1"):
+                app.button1.setVisible(not has_data)
+            if hasattr(app, "button3"):
+                app.button3.setVisible(has_data)
+
+            
+            self.add_to_recent_projects(os.path.normpath(filename))
+
+            # Reset baselines cleanly
+            self.reset_all_dirty_states()
+
+            QMessageBox.information(app, "Loaded", f"Project loaded from:\n{filename}")
+
+        except Exception as e:
+            QMessageBox.critical(app, "Error", f"Failed to load project:\n{e}")
+
+    # =========================================================================
+    # Project Saving (Save / Save As)
+    # =========================================================================
+    def save_project(self, onCrash=False, ask=False):
+        """
+        Saves the project to disk.
+        - ask=False: Quick Save to self.app.current_project_path (prompts if unsaved).
+        - ask=True: Save As (always prompts for location).
+        """
+        app = self.app
+
+        # 1. Sync active text fields into active metadata dict
+        if hasattr(app, "current_file") and app.current_file and hasattr(app, "meta_fields"):
+            if not hasattr(app, "file_metadata"):
+                app.file_metadata = {}
+            app.file_metadata[app.current_file] = {
+                key: field.text().strip() for key, field in app.meta_fields.items()
+            }
+
+        # 2. Check if saveable state exists
+        has_files = len(getattr(app, "selected_paths", [])) > 0
+        has_metadata = any(
+            any(val for val in m.values()) for m in getattr(app, "file_metadata", {}).values()
+        )
+        has_param_changes = any(
+            s.has_any_changes() for s in getattr(app, "param_sections", [])
+        )
+        has_processed_data = any(
+            len(getattr(app, item["key"], {})) > 0 for item in DATA_SCHEMA
+        )
+
+        if not (has_files or has_processed_data or has_metadata or has_param_changes):
+            if not onCrash:
+                QMessageBox.warning(
+                    app,
+                    "Save Project",
+                    "There is no data or configuration to save.",
+                )
+            return
+
+        # 3. Path Resolution
+        filename = None
+        if not onCrash:
+            existing_path = getattr(app, "current_project_path", None)
+            if ask or not existing_path:
+                start_dir = existing_path if existing_path else ""
+                filename, _ = QFileDialog.getSaveFileName(
+                    app, "Save Project", start_dir, "FLARE Project (*.flare)"
+                )
+                if not filename:
+                    return
+            else:
+                filename = existing_path
+        else:
+            if PLATFORM_NAME == "darwin":
+                filename = os.path.join(
+                    os.path.dirname(sys.executable), "../../../flares_autosave.flare"
+                )
+            else:
+                filename = os.path.join(os.getcwd(), "flares_autosave.flare")
+
+        try:
+            if not filename.endswith(".flare"):
+                filename += ".flare"
+
+            project_path = Path(filename).resolve()
+            project_dir = project_path.parent
+
+            # 4. Convert Paths to Relative
+            bubble_widgets = getattr(app, "bubble_widgets", {})
+            file_list = [
+                self.get_safe_path(b.file_path, project_dir) for b in bubble_widgets.values()
+            ]
+            progress_states = {
+                self.get_safe_path(b.file_path, project_dir): getattr(b, "current_step", 0)
+                for b in bubble_widgets.values()
+            }
+
+            rel_metadata = {}
+            for full_path, meta in getattr(app, "metadata_cache", {}).items():
+                try:
+                    rel_metadata[self.get_safe_path(full_path, project_dir)] = meta
+                except Exception as e:
+                    print(f"Metadata conversion failed for {full_path}: {e}")
+
+            rel_file_params = {
+                self.get_safe_path(f_path, project_dir): meta
+                for f_path, meta in getattr(app, "file_metadata", {}).items()
+            }
+
+            current_params = app.get_all_current_ui_params()
+            if not current_params and getattr(app, "config_dict", None):
+                first_file = next(iter(app.config_dict.keys()))
+                current_params = app.config_dict[first_file]
+
+            # 5. Build Serialized Payload
+            project_data = {
+                item["key"]: getattr(app, item["key"], {}) for item in DATA_SCHEMA
+            }
+            project_data.update({
+                "version": CURRENT_VERSION,
+                "file_list": file_list,
+                "progress_states": progress_states,
+                "file_metadata": rel_metadata,
+                "file_parameters": rel_file_params,
+                "current_ui_params": current_params,
+            })
+
+            def sanitize(obj):
+                if isinstance(obj, Path):
+                    return str(PurePosixPath(obj))
+                elif isinstance(obj, dict):
+                    return {sanitize(k): sanitize(v) for k, v in obj.items()}
+                elif isinstance(obj, list):
+                    return [sanitize(i) for i in obj]
+                return obj
+
+            project_data = sanitize(project_data)
+
+            self.add_to_recent_projects(os.path.normpath(filename))
+
+            # 6. Background Saving Execution
+            if not onCrash:
+                app.saving_overlay = SavingOverlay(app)
+                app.saving_overlay.resize(app.size())
+                app.saving_overlay.show()
+
+            app.save_thread = SaveProjectThread(filename, project_data)
+
+            def _on_save_success(saved_file):
+                if hasattr(app, "saving_overlay"):
+                    app.saving_overlay.close()
+
+                self.add_to_recent_projects(os.path.normpath(saved_file))
+
+                app.current_project_path = saved_file
+                self.reset_all_dirty_states()
+
+                if not onCrash:
+                    QMessageBox.information(
+                        app, "Success", f"Project saved to:\n{saved_file}"
+                    )
+
+            def _on_save_error(error_msg):
+                if hasattr(app, "saving_overlay"):
+                    app.saving_overlay.close()
+                if not onCrash:
+                    QMessageBox.critical(
+                        app, "Error", f"Failed to save project:\n{error_msg}"
+                    )
+
+            app.save_thread.finished_signal.connect(_on_save_success)
+            app.save_thread.error_signal.connect(_on_save_error)
+            app.save_thread.start()
+
+        except Exception as e:
+            if not onCrash:
+                QMessageBox.critical(app, "Error", f"Failed to save project:\n{e}")
+
+
+    def update_recent_projects_menu(self):
+        """Clears and rebuilds the Recent Projects submenu items."""
+        app = self.app
+        if not hasattr(app, "recent_projects_menu"):
+            return
+
+        app.recent_projects_menu.clear()
+
+        raw_projects = self.file_cfg.get("File", "recent_projects", fallback="")
+        projects = [p.strip() for p in raw_projects.split(",") if p.strip()]
+
+        if not projects:
+            no_recent = app.recent_projects_menu.addAction(
+                "No Recent Projects"
+            )
+            no_recent.setEnabled(False)
+            return
+
+        for i, project_path in enumerate(projects):
+            action = QAction(f"{i+1}: {project_path}", app)
+            action.setToolTip(project_path)
+            action.triggered.connect(
+                lambda checked, path=project_path: self.open_recent_project(path)
+            )
+            app.recent_projects_menu.addAction(action)
+
+    def add_to_recent_projects(self, project_path):
+        """Adds a project path, moves it to the top, and hard caps at 10."""
+        raw_projects = self.file_cfg.get("File", "recent_projects", fallback="")
+        projects = [p.strip() for p in raw_projects.split(",") if p.strip()]
+
+        if project_path in projects:
+            projects.remove(project_path)
+
+        projects.insert(0, project_path)
+        projects = projects[:10]  # Hard cap of 10 items
+
+        self.file_cfg.set("File", "recent_projects", ",".join(projects))
+        try:
+            with open(self.cfg_path, "w") as f:
+                self.file_cfg.write(f)
+        except Exception as e:
+            print(f"Warning: Could not save config history: {e}")
+
+        self.update_recent_projects_menu()
+
+    def open_recent_project(self, project_path):
+        """The slot that executes when a recent project entry is clicked."""
+        if os.path.exists(project_path):
+            print(f"Opening recent project: {project_path}")
+
+            # Route project loading through ProjectManager or app's loader
+            if hasattr(self.app, "project_loader"):
+                self.app.project_loader(project_path)
+            else:
+                self.load_project(project_path)
+
+            self.add_to_recent_projects(project_path)
+        else:
+            QMessageBox.warning(
+                self.app,
+                "Project Not Found",
+                f"The project file could not be found:\n{project_path}",
+            )
+            # Clean out the broken path
+            raw_projects = self.file_cfg.get("File", "recent_projects", fallback="")
+            projects = [
+                p.strip()
+                for p in raw_projects.split(",")
+                if p.strip() and p.strip() != project_path
+            ]
+            self.file_cfg.set("File", "recent_projects", ",".join(projects))
+            self.update_recent_projects_menu()
+
+    # =========================================================================
+    # Recent Files Operations
+    # =========================================================================
+    def update_recent_files_menu(self):
+        """Clears and rebuilds the Recent Files submenu items."""
+        app = self.app
+        if not hasattr(app, "recent_files_menu"):
+            return
+
+        app.recent_files_menu.clear()
+
+        raw_files = self.file_cfg.get("File", "recent_files", fallback="")
+        files = [f.strip() for f in raw_files.split(",") if f.strip()]
+
+        if not files:
+            no_recent = app.recent_files_menu.addAction("No Recent Files")
+            no_recent.setEnabled(False)
+            return
+
+        for i, file_path in enumerate(files):
+            action = QAction(f"{i+1}: {file_path}", app)
+            action.triggered.connect(
+                lambda checked, path=file_path: self.open_recent_file(path)
+            )
+            app.recent_files_menu.addAction(action)
+
+    def add_to_recent_files(self, file_path):
+        """Adds a path, moves it to the top, and hard caps the list at 10."""
+        raw_files = self.file_cfg.get("File", "recent_files", fallback="")
+        files = [f.strip() for f in raw_files.split(",") if f.strip()]
+
+        if file_path in files:
+            files.remove(file_path)
+
+        files.insert(0, file_path)
+        files = files[:10]
+
+        self.file_cfg.set("File", "recent_files", ",".join(files))
+        try:
+            with open(self.cfg_path, "w") as f:
+                self.file_cfg.write(f)
+        except Exception as e:
+            print(f"Warning: Could not save config history: {e}")
+
+        self.update_recent_files_menu()
+
+    def open_recent_file(self, file_path):
+        """The slot that executes when someone clicks a recent file entry."""
+        if os.path.exists(file_path):
+            print(f"Opening recent file: {file_path}")
+            self._load_files_into_pipeline([os.path.normpath(file_path)])
+
+            # Refresh position to top
+            self.add_to_recent_files(file_path)
+        else:
+            QMessageBox.warning(
+                self.app,
+                "File Not Found",
+                f"The file could not be found:\n{file_path}",
+            )
+            # Clean up the broken link from history
+            raw_files = self.file_cfg.get("File", "recent_files", fallback="")
+            files = [
+                f.strip()
+                for f in raw_files.split(",")
+                if f.strip() and f.strip() != file_path
+            ]
+            self.file_cfg.set("File", "recent_files", ",".join(files))
+            self.update_recent_files_menu()
+
+    # =========================================================================
+    # Baseline Synchronization & Dirty Checks
+    # =========================================================================
+    def sync_metadata_baseline(self):
+        """Captures current file_metadata state as baseline."""
+        self.app.saved_file_metadata = copy.deepcopy(getattr(self.app, "file_metadata", {}))
+
+    def is_metadata_dirty(self):
+        """Returns True if file metadata has been modified relative to saved baseline."""
+        current_meta = getattr(self.app, "file_metadata", {})
+        saved_meta = getattr(self.app, "saved_file_metadata", {})
+
+        if current_meta != saved_meta:
+            return True
+
+        if (
+            hasattr(self.app, "current_file")
+            and self.app.current_file
+            and hasattr(self.app, "meta_fields")
+        ):
+            active_saved = saved_meta.get(self.app.current_file, {})
+            for key, field in getattr(self.app, "meta_fields", {}).items():
+                if field.text().strip() != active_saved.get(key, "").strip():
+                    return True
+
+        return False
+
+    def reset_all_dirty_states(self):
+        """Resets parameter, file, and metadata baselines after load/save."""
+        # 1. Sync file list baseline
+        self.app.saved_selected_paths = copy.deepcopy(getattr(self.app, "selected_paths", []))
+        self.app.files_are_dirty = False
+
+        # 2. Sync metadata baseline
+        self.sync_metadata_baseline()
+
+        # 3. Sync parameter baselines
+        for section_widget in getattr(self.app, "param_sections", []):
+            if hasattr(section_widget, "save_current_as_baseline"):
+                section_widget.save_current_as_baseline()
+
+        # 4. Clear dirty status & update UI title
+        self.app.is_saved = True
+        if hasattr(self.app, "update_window_title"):
+            self.app.update_window_title()
+
+
+
+
+def _get_bids_demographics(snirf_path: str) -> dict[str, str]:
+    """Traverses the path of a SNIRF file to extract age/sex/hand from BIDS TSV files.
+    'hand' is only included if a value is present and isn't 'n/a' (case-insensitive) -
+    many datasets leave it unset/inapplicable, so surfacing 'n/a' explicitly just adds noise.
+    """
+    path = Path(snirf_path)
+    fields = ["age", "sex", "hand"]
+
+    # Extract sub-XX and ses-YY labels from the path
+    sub_id = next((part for part in path.parts if part.startswith("sub-")), None)
+    ses_id = next((part for part in path.parts if part.startswith("ses-")), None)
+
+    if not sub_id:
+        return {}
+
+    def _row_to_dict(row) -> dict[str, str]:
+        result = {}
+        for field in fields:
+            if field not in row:
+                continue
+            val = row[field]
+            if pd.isna(val):
+                continue
+            val_str = str(val).strip()
+            if field == "hand" and val_str.lower() in ("n/a", "na", ""):
+                continue
+            result[field] = val_str
+        return result
+
+    # 1. Look for sub-<id>/sub-<id>_sessions.tsv
+    sub_dir = next((p for p in path.parents if p.name == sub_id), None)
+    if sub_dir and ses_id:
+        sessions_tsv = sub_dir / f"{sub_id}_sessions.tsv"
+        if sessions_tsv.exists():
+            try:
+                df = pd.read_csv(sessions_tsv, sep="\t")
+                matching = df[df["session_id"].astype(str).str.replace("ses-", "") == ses_id.replace("ses-", "")]
+                if not matching.empty:
+                    result = _row_to_dict(matching.iloc[0])
+                    if result:
+                        return result
+            except Exception:
+                pass
+
+    # 2. Fallback: Check dataset root participants.tsv
+    bids_root = sub_dir.parent if sub_dir else None
+    if bids_root:
+        participants_tsv = bids_root / "participants.tsv"
+        if participants_tsv.exists():
+            try:
+                df = pd.read_csv(participants_tsv, sep="\t")
+                matching = df[df["participant_id"].astype(str).str.replace("sub-", "") == sub_id.replace("sub-", "")]
+                if not matching.empty:
+                    result = _row_to_dict(matching.iloc[0])
+                    if result:
+                        return result
+            except Exception:
+                pass
+
+    return {}
+
+
+
+def extract_metadata_worker(file_name):
+    """Runs in the separate worker process. Returns a clean dict."""
+
+    # 1. Use preload=False! We only need metadata.
+    raw = None
+
+    try:
+        raw = read_raw_snirf(file_name, preload=False, verbose="ERROR")
+        snirf_info = {}
+
+        # 2. Measurement date
+        snirf_info['Measurement Date'] = str(raw.info.get('meas_date'))
+
+        # 3. Short Channels
+        try:
+            short_chans = get_short_channels(raw, max_dist=0.015)
+            names = list(short_chans.ch_names)
+            snirf_info['Short Channels'] = f"Likely - {names}"
+            total_chans = len(raw.ch_names)
+            pct_short = (len(names) / total_chans * 100) if total_chans else 0
+            if pct_short > 25:
+                snirf_info['Short Channels'] += "\n There are a lot of short channels. Optode distances are likely incorrect!"
+        except:
+            snirf_info['Short Channels'] = "Unlikely"
+
+        # 4. Distances
+        dist_vals = source_detector_distances(raw.info)
+        snirf_info['Source-Detector Distances'] = [
+            f"{name}: {d:.4f} m" for name, d in zip(raw.info['ch_names'], dist_vals)
+        ]
+
+        # 5. Digitization
+        dig = raw.info.get('dig', None)
+        if dig is not None:
+            snirf_info['Digitization Points'] = [
+                f"Kind: {p['kind']}, ID: {p['ident']}, Coord: {p['r']}" for p in dig
+            ]
+        else:
+            snirf_info['Digitization Points'] = "Not found"
+
+        # 6. Annotations (using our copy-to-string trick)
+        if raw.annotations is not None and len(raw.annotations) > 0:
+            snirf_info['Annotations'] = [
+                f"Onset: {o:.2f}s, Duration: {d:.2f}s, Description: {str(desc)}"
+                for o, d, desc in zip(raw.annotations.onset, raw.annotations.duration, raw.annotations.description)
+            ]
+        else:
+            snirf_info['Annotations'] = "No annotations found"
+
+        demographics = _get_bids_demographics(file_name)
+        if "age" in demographics:
+            snirf_info["BIDS - Age"] = demographics["age"]
+        if "sex" in demographics:
+            snirf_info["BIDS - Sex"] = demographics["sex"]
+        if "hand" in demographics:
+            snirf_info["BIDS - Handedness"] = demographics["hand"]
+        return snirf_info
+    
+    except Exception as e:
+        print(f"Worker safely caught failure on {file_name}: {str(e)}")
+        return {'status': 'error', 'reason': str(e)}
+
+    finally:
+        if raw is not None:
+            try:
+                raw.close()
+            except:
+                pass

src/shared/flaresbasewidget.py

@@ -7,18 +7,17 @@ License: GPL-3.0
 """
 
 import os
-import json
-
-from pathlib import Path
+from copy import deepcopy
 from typing import Sequence, Any
+
 import pandas as pd
 from pandas import DataFrame
-from PySide6.QtWidgets import QApplication, QComboBox, QDialog, QGridLayout, QHBoxLayout, QLabel, QLineEdit, QListView, QMessageBox, QPushButton, QScrollArea, QVBoxLayout, QWidget, QFrame, QSpinBox
+
+from PySide6.QtWidgets import QApplication, QComboBox, QDialog, QGridLayout, QHBoxLayout, QLabel, QLineEdit, QListView, QMessageBox, QPushButton, QScrollArea, QSizePolicy, QVBoxLayout, QWidget, QFrame, QSpinBox, QFileDialog
 from PySide6.QtGui import QStandardItemModel, QStandardItem, QPixmap, QIntValidator, QDoubleValidator
-from PySide6.QtCore import QEvent, QSize, Qt
-
-from src.shared.shareddata import APP_NAME
+from PySide6.QtCore import QEvent, QPoint, QSize, QTimer, Qt, Signal
 
+from src.shared.shareddata import APP_NAME, PIPELINE_STAGES
 
 class FullClickComboBox(QComboBox):
     def __init__(self, parent=None):
@@ -51,7 +50,7 @@ class ClickableLabel(QLabel):
     
     def mousePressEvent(self, event):
         viewer = QWidget()
-        viewer.setWindowTitle("Expanded View")
+        viewer.setWindowTitle(f"Expanded View" - {APP_NAME.upper()})
         layout = QVBoxLayout(viewer)
         label = QLabel()
         label.setPixmap(self._pixmap_full)
@@ -85,7 +84,7 @@ class ParameterInputDialog(QDialog):
         }
         """
         super().__init__(parent)
-        self.setWindowTitle("Input Parameters")
+        self.setWindowTitle(f"Input Parameters - {APP_NAME.upper()}")
         self.params_dict = params_dict
         self.inputs = {}  # {(idx, param_key): QLineEdit}
 
@@ -218,7 +217,7 @@ class ParameterInputDialog(QDialog):
     def _show_error(self, message):
         error_box = QMessageBox(self)
         error_box.setIcon(QMessageBox.Critical)
-        error_box.setWindowTitle("Input Error")
+        error_box.setWindowTitle(f"Input Error - {APP_NAME.upper()}")
         error_box.setText(message)
         error_box.exec_()
 
@@ -244,9 +243,6 @@ class FullClickComboBox(QComboBox):
 
 
 
-from PySide6.QtWidgets import QWidget, QHBoxLayout, QLineEdit, QPushButton, QFileDialog
-from PySide6.QtCore import Signal
-
 class FilePickerWidget(QWidget):
     # This custom signal lets our container mimic a standard QLineEdit
     textChanged = Signal(str)
@@ -294,9 +290,7 @@ class FilePickerWidget(QWidget):
 
 class ParamSection(QWidget):
     """
-    A widget section that dynamically creates labeled input fields from parameter metadata.
-
-    Args:
+        Args:
         section_data (dict): Dictionary containing section title and list of parameter info.
             Expected format:
                 {
@@ -312,6 +306,8 @@ class ParamSection(QWidget):
                     ]
                 }
     """
+
+    dirty_state_changed = Signal(bool)
         
     def __init__(self, section_data, global_widgets):
         super().__init__()
@@ -323,6 +319,8 @@ class ParamSection(QWidget):
 
         self.param_rows = [] 
         self.header_widgets = []
+        self.dirty_params = {}
+        self._updating_checkstates = False
 
         # Title label
         title_label = QLabel(section_data["title"])
@@ -339,30 +337,23 @@ class ParamSection(QWidget):
 
         for param in section_data["params"]:
             h_layout = QHBoxLayout()
-
             is_advanced = param.get("advanced", False)
+            param_name = param["name"]
+            help_text = param.get("help", "")
 
             # Build label text and tooltips
-            param_name = param["name"]
-            if is_advanced:
-                label_text = f"⚠️ {param_name}"
-            else:
-                label_text = param_name
-
+            label_text = f"⚠️ {param_name}" if is_advanced else param_name
             label = QLabel(label_text)
-
-            # Set hover tooltip
+            
             if is_advanced:
-                label.setToolTip(f"ADVANCED: {param.get("help", "")}")
+                label.setToolTip(f"ADVANCED: {help_text}")
             else:
-                label.setToolTip(param.get("help", ""))
-
-            help_text = param.get("help", "")
+                label.setToolTip(help_text)
 
             help_btn = QPushButton("?")
             help_btn.setFixedWidth(25)
             help_btn.setToolTip(help_text)
-            help_btn.clicked.connect(lambda _, text=help_text: self.show_help_popup(text))
+            help_btn.clicked.connect(lambda _, text=help_text, name=param_name: self.show_help_popup(name, text))
 
             h_layout.addWidget(help_btn)
             h_layout.addWidget(label)
@@ -370,220 +361,331 @@ class ParamSection(QWidget):
             h_layout.setStretch(1, 6)
 
             default_val = param["default"]
+            param_type = param["type"]
+            type_str = str(param_type).lower()
 
             # Create input widget based on type
-            if param["type"] == bool:
+            if param_type == bool or "bool" in type_str:
                 widget = QComboBox()
                 widget.addItems(["True", "False"])
                 widget.setCurrentText(str(default_val))
-                widget.currentTextChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
+                widget.currentTextChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
                 widget.currentTextChanged.connect(self.notify_global_update)
-            elif param["type"] == int:
+                
+            elif param_type == int or "int" in type_str:
                 widget = QLineEdit()
                 widget.setValidator(QIntValidator())
                 widget.setText(str(default_val))
-                widget.textChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
-            elif param["type"] == float:
+                widget.textChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
+                
+            elif param_type == float or "float" in type_str:
                 widget = QLineEdit()
                 widget.setValidator(QDoubleValidator())
                 widget.setText(str(default_val))
-                widget.textChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
-            elif param["type"] == list:
+                widget.textChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
+                
+            elif param_type == list or "list" in type_str:
+                options = param.get("options", [])
                 if param.get("exclusive", True):
                     widget = QComboBox()
-                    widget.addItems(param.get("options", []))
-                    widget.setCurrentText(str(default_val))
-                    widget.currentTextChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
+                    widget.addItems(options)
+                    initial_text = default_val[0] if (isinstance(default_val, list) and len(default_val) > 0) else str(default_val)
+                    widget.setCurrentText(initial_text)
+                    widget.currentTextChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
                     widget.currentTextChanged.connect(self.notify_global_update)
                 else:
-                    widget = self._create_multiselect_dropdown(None)
-            elif param["type"] == range:
+                    widget = self._create_multiselect_dropdown(options, default_val=default_val, param_name=param_name)
+
+            elif param_type == range or "range" in type_str:
                 widget = QSpinBox()
-                widget.setRange(0, 999)  #NOTE: will this be a high enough limit?
-                # If default is "None" or range(15), handle it gracefully:
+                widget.setRange(0, 999)
                 if isinstance(default_val, range):
                     widget.setValue(default_val.stop)
                 elif str(default_val).isdigit():
                     widget.setValue(int(default_val))
                 else:
-                    widget.setValue(15) # Default fallback
-                widget.valueChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
-            elif param["type"] == "json_file":
-                # Create our custom dual-element compound layout widget
+                    widget.setValue(15)
+                widget.valueChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
+                
+            elif param_type == "json_file":
                 widget = FilePickerWidget(default_val=default_val, file_filter="JSON Files (*.json)")
-                # Your existing tracking signals connect seamlessly!
-                widget.textChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
+                widget.textChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
+                
             else: 
                 widget = QLineEdit()
                 widget.setText(str(default_val))
-                widget.textChanged.connect(lambda val, p=param["name"]: self.check_if_changed(p, val))
+                widget.textChanged.connect(lambda _, p=param_name: self.check_if_changed(p))
+
+            self.widgets[param_name] = {
+                "widget": widget,
+                "label": label,
+                "default": default_val,
+                "saved_value": deepcopy(default_val),
+                "type": param_type,
+                "h_layout": h_layout
+            }
 
             if "depends_on" in param:
-                deps_list = []
-                
-                if isinstance(param["depends_on"], list):
-                    deps_list = param["depends_on"]
-                
-                else:
-                    deps_list.append({
-                        "parent_name": param["depends_on"],
-                        "depends_value": param.get("depends_value", "True")
-                    })
-
+                deps_list = param["depends_on"] if isinstance(param["depends_on"], list) else [{
+                    "parent_name": param["depends_on"],
+                    "depends_value": param.get("depends_value", "True")
+                }]
                 self.dependencies.append({
-                    "child_name": param["name"],
+                    "child_name": param_name,
                     "conditions": deps_list
                 })
 
             widget.setToolTip(help_text)
-         
             h_layout.addWidget(widget)
             h_layout.setStretch(2, 3)
 
             layout.addLayout(h_layout)
-            self.widgets[param["name"]] = {
-                "widget": widget,
-                "label": label,
-                "default": default_val,
-                "type": param["type"],
-                "h_layout": h_layout
-            }
-
-            row_widgets = [help_btn, label, widget]
-            is_advanced = param.get("advanced", False)
-            self.param_rows.append((row_widgets, h_layout, is_advanced))
+            self.param_rows.append(([help_btn, label, widget], h_layout, is_advanced))
 
         self.update_dependencies()
 
-    def set_advanced_visible(self, show_advanced: bool):
-        """Shows/hides advanced parameters and adjusts header visibility dynamically."""
-        has_visible_rows = False
+    def is_different(self, val_a, val_b, param_type=None):
+        """Compares two parameter values to determine if they differ."""
+        type_str = str(param_type).lower()
 
-        for row_widgets, h_layout, is_advanced in self.param_rows:
-            visible = show_advanced or not is_advanced
-            
-            # Toggle visibility of all child widgets in the row
-            for w in row_widgets:
-                w.setVisible(visible)
-            
-            if visible:
-                has_visible_rows = True
+        # BOOL comparison
+        if param_type == bool or "bool" in type_str:
+            def to_bool(v):
+                if isinstance(v, bool):
+                    return v
+                return str(v).strip().lower() in ("true", "1", "yes")
+            return to_bool(val_a) != to_bool(val_b)
 
-        # Hide or show section header title and horizontal rule if section is completely empty
-        for hw in self.header_widgets:
-            hw.setVisible(has_visible_rows)
-            
-        self.setVisible(has_visible_rows)
+        # LIST comparison
+        if param_type == list or "list" in type_str:
+            def to_list(v):
+                if v is None:
+                    return []
+                if isinstance(v, list):
+                    res = []
+                    for item in v:
+                        res.extend(to_list(item))
+                    return res
+                if isinstance(v, str):
+                    s = v.strip()
+                    if s.startswith('[') and s.endswith(']'):
+                        s = s[1:-1]
+                    items = [x.strip().strip("'\"") for x in s.split(',') if x.strip().strip("'\"")]
+                    return [i for i in items if i != "<None Selected>"]
+                return [str(v).strip()]
 
+            return sorted(to_list(val_a)) != sorted(to_list(val_b))
 
-    def has_any_changes(self):
-        """Returns True if any parameter in this section differs from its default."""
-        for name, info in self.widgets.items():
-            default = info["default"]
-            
-            current_val = self.get_param_values().get(name)
-            
-            if str(current_val) != str(default):
-                return True
-        return False
-    
+        # RANGE comparison
+        if param_type == range or "range" in type_str:
+            def to_range_stop(v):
+                if isinstance(v, range):
+                    return v.stop
+                try:
+                    return int(v)
+                except (ValueError, TypeError):
+                    return 0
+            return to_range_stop(val_a) != to_range_stop(val_b)
 
-    def check_if_changed(self, param_name, current_value):
-        """Toggles bold font on the label if the value differs from default."""
+        # INT / FLOAT comparison
+        if param_type in (int, float) or "int" in type_str or "float" in type_str:
+            try:
+                if val_a is not None and val_b is not None and str(val_a).strip() != "" and str(val_b).strip() != "":
+                    return float(val_a) != float(val_b)
+            except (ValueError, TypeError):
+                pass
+
+        # String / Default Fallback
+        str_a = "" if val_a is None else str(val_a).strip()
+        str_b = "" if val_b is None else str(val_b).strip()
+        return str_a != str_b
+
+    def check_if_changed(self, param_name, current_value=None, *args, **kwargs):
+        """
+        Toggles bold font/blue color on the label if the value differs from default.
+        Always pulls current UI value from get_param_values() if not explicitly passed.
+        """
         info = self.widgets.get(param_name)
         if not info:
             return
 
         label = info["label"]
         default = info["default"]
-        
-        is_changed = False
-        
-        if info["type"] == list:
-            # If it's an exclusive ComboBox, current_value is a string.
-            # We wrap it in a list to compare it to the default list.
-            if isinstance(current_value, str):
-                normalized_current = [current_value]
-            else:
-                normalized_current = current_value # Already a list from multi-select
-            
-            # Ensure default is a list for comparison
-            normalized_default = default if isinstance(default, list) else [default]
-            
-            # Use sorted to ensure order doesn't matter
-            is_changed = sorted(normalized_current) != sorted(normalized_default)
+        saved = info.get("saved_value", default)
+        param_type = info["type"]
 
-        # 2. Handle Range (SpinBox)
-        elif info["type"] == range:
-            ref = default.stop if isinstance(default, range) else default
-            try:
-                is_changed = int(current_value) != int(ref)
-            except (ValueError, TypeError):
-                is_changed = True
+        if current_value is None:
+            current_value = self.get_param_values().get(param_name)
 
-        # 3. Standard Comparison (bool, int, float, str)
-        else:
-            is_changed = str(current_value) != str(default)
-
-        if is_changed:
-            label.setStyleSheet("color: #3498db; font-weight: bold;") # Nice Blue
+        # 1. COLORING LOGIC (Compares against 'default')
+        if self.is_different(current_value, default, param_type):
+            label.setStyleSheet("color: #3498db; font-weight: bold;")
         else:
             label.setStyleSheet("")
 
+        # 2. DIRTY STATE LOGIC (Compares against 'saved_value')
+        self.dirty_params[param_name] = self.is_different(current_value, saved, param_type)
+        self.dirty_state_changed.emit(any(self.dirty_params.values()))
+
+    def reset_to_defaults(self):
+        """
+        Resets all UI input widgets back to their defined default values in SECTIONS,
+        resets saved baseline states, updates label styling (removes blue highlight),
+        and updates dependent widget visibility/enablement.
+        """
+        for param_name, info in self.widgets.items():
+            widget = info["widget"]
+            default_val = info["default"]
+            param_type = info["type"]
+            type_str = str(param_type).lower()
+
+            # Update saved baseline to match default
+            info["saved_value"] = deepcopy(default_val)
+
+            # Reset Widget Values
+            if param_type == bool or "bool" in type_str:
+                if isinstance(widget, QComboBox):
+                    widget.setCurrentText(str(default_val))
+
+            elif param_type in (int, float, str) or "int" in type_str or "float" in type_str or "str" in type_str:
+                if isinstance(widget, QLineEdit):
+                    widget.setText("" if default_val is None else str(default_val))
+
+            elif param_type == list or "list" in type_str:
+                if isinstance(widget, FullClickComboBox):
+                    defaults = default_val if isinstance(default_val, list) else ([default_val] if default_val else [])
+                    model = widget.model()
+                    self._updating_checkstates = True
+                    normal_items = []
+                    for i in range(2, model.rowCount()):
+                        item = model.item(i)
+                        normal_items.append(item)
+                        state = Qt.Checked if item.text() in defaults else Qt.Unchecked
+                        item.setCheckState(state)
+                    
+                    # Sync toggle select all item
+                    toggle_item = model.item(1)
+                    if toggle_item and normal_items:
+                        all_checked = all(i.checkState() == Qt.Checked for i in normal_items)
+                        toggle_item.setCheckState(Qt.Checked if all_checked else Qt.Unchecked)
+
+                    self._updating_checkstates = False
+                    self.update_dropdown_label(param_name)
+                elif isinstance(widget, QComboBox):
+                    initial_text = default_val[0] if (isinstance(default_val, list) and len(default_val) > 0) else str(default_val)
+                    widget.setCurrentText(initial_text)
+
+            elif param_type == range or "range" in type_str:
+                if isinstance(widget, QSpinBox):
+                    if isinstance(default_val, range):
+                        widget.setValue(default_val.stop)
+                    elif str(default_val).isdigit():
+                        widget.setValue(int(default_val))
+                    else:
+                        widget.setValue(15)
+
+            elif param_type == "json_file":
+                if hasattr(widget, "setText"):
+                    widget.setText("" if default_val is None else str(default_val))
+
+            # Re-evaluate visual styling and dirty flags
+            self.check_if_changed(param_name)
+
+        self.dirty_params.clear()
+        self.dirty_state_changed.emit(False)
+        self.update_dependencies()
+
+    # Aliases for clear button calls
+    def clear(self):
+        self.reset_to_defaults()
+
+    def reset(self):
+        self.reset_to_defaults()
+
+    def reset_baseline_to_default(self):
+        """Resets baseline saved values back to defaults and re-checks visual styling."""
+        for name, info in self.widgets.items():
+            info["saved_value"] = deepcopy(info["default"])
+            self.check_if_changed(name)
+
+        self.dirty_params.clear()
+        self.dirty_state_changed.emit(False)
+
+    def save_current_as_baseline(self):
+        """Call this when a project is saved to lock current UI state as saved_value."""
+        current_values = self.get_param_values()
+        for name, info in self.widgets.items():
+            if name in current_values:
+                info["saved_value"] = deepcopy(current_values[name])
+            self.check_if_changed(name)
+
+        self.dirty_params.clear()
+        self.dirty_state_changed.emit(False)
+
+    def set_advanced_visible(self, show_advanced: bool):
+        has_visible_rows = False
+        for row_widgets, _, is_advanced in self.param_rows:
+            visible = show_advanced or not is_advanced
+            for w in row_widgets:
+                w.setVisible(visible)
+            if visible:
+                has_visible_rows = True
+
+        for hw in self.header_widgets:
+            hw.setVisible(has_visible_rows)
+            
+        self.setVisible(has_visible_rows)
+
+    def has_any_changes(self):
+        current_values = self.get_param_values()
+        for name, info in self.widgets.items():
+            if self.is_different(current_values.get(name), info["default"], info["type"]):
+                return True
+        return False
+
     def notify_global_update(self):
-        """
-        Since dependencies can cross sections, we need to tell 
-        all sections to refresh their enabled/disabled states.
-        """
-        # If you have a reference to the parent container, call its update.
-        # Otherwise, you can iterate through the known param_sections:
-        for section in self.parent().findChildren(ParamSection):
-            section.update_dependencies()
+        parent = self.parent()
+        if parent:
+            for section in parent.findChildren(ParamSection):
+                section.update_dependencies()
 
     def update_dependencies(self):
-        """Disables/Enables widgets based on parent selection values."""
         for dep in self.dependencies:
             child_info = self.widgets.get(dep["child_name"])
             if not child_info:
                 continue
 
-            # Default to enabled until a condition fails
             all_conditions_met = True
-
             for cond in dep["conditions"]:
                 parent_name = cond.get("parent_name") or cond.get("parent")
-                required_value = str(cond.get("depends_value") if "depends_value" in cond else cond.get("value", "True"))
+                required_val = str(cond.get("depends_value") if "depends_value" in cond else cond.get("value", "True"))
                 
                 parent_info = self.widgets.get(parent_name)
                 if not parent_info:
                     all_conditions_met = False
                     break
 
-                parent_widget = parent_info["widget"]
-
-                # Extract current parent value based on widget type
-                if isinstance(parent_widget, QComboBox):
-                    current_parent_value = parent_widget.currentText()
-                elif isinstance(parent_widget, QLineEdit):
-                    current_parent_value = parent_widget.text()
-                elif isinstance(parent_widget, QSpinBox):
-                    current_parent_value = str(parent_widget.value())
+                p_widget = parent_info["widget"]
+                if isinstance(p_widget, QComboBox):
+                    curr_val = p_widget.currentText()
+                elif isinstance(p_widget, QLineEdit):
+                    curr_val = p_widget.text()
+                elif isinstance(p_widget, QSpinBox):
+                    curr_val = str(p_widget.value())
                 else:
-                    current_parent_value = str(parent_widget)
+                    curr_val = str(p_widget)
 
-                # If any condition fails, flag as false
-                if current_parent_value != required_value:
+                if curr_val != required_val:
                     all_conditions_met = False
                     break
-                
-            # Toggle the entire row (Button, Label, and Input)
+
             h_layout = child_info["h_layout"]
             for i in range(h_layout.count()):
                 item = h_layout.itemAt(i).widget()
                 if item:
                     item.setEnabled(all_conditions_met)
 
-    def _create_multiselect_dropdown(self, items):
+    def _create_multiselect_dropdown(self, items, default_val=None, param_name=""):
         combo = FullClickComboBox()
         combo.setView(QListView())
         model = QStandardItemModel()
@@ -601,16 +703,18 @@ class ParamSection(QWidget):
         toggle_item.setData(Qt.Unchecked, Qt.CheckStateRole)
         model.appendRow(toggle_item)
 
-        if items is not None:
-            for item in items:
-                standard_item = QStandardItem(item)
+        defaults = default_val if isinstance(default_val, list) else ([default_val] if default_val else [])
+
+        if items:
+            for item_text in items:
+                standard_item = QStandardItem(item_text)
                 standard_item.setFlags(Qt.ItemIsUserCheckable | Qt.ItemIsEnabled)
-                standard_item.setData(Qt.Unchecked, Qt.CheckStateRole)
+                initial_state = Qt.Checked if item_text in defaults else Qt.Unchecked
+                standard_item.setData(initial_state, Qt.CheckStateRole)
                 model.appendRow(standard_item)
 
         combo.setInsertPolicy(QComboBox.NoInsert)
 
-
         def on_view_clicked(index):
             item = model.itemFromIndex(index)
             if item.isCheckable():
@@ -619,171 +723,70 @@ class ParamSection(QWidget):
 
         combo.view().pressed.connect(on_view_clicked)
 
-        self._updating_checkstates = False
-
         def on_item_changed(item):
             if self._updating_checkstates:
                 return
             self._updating_checkstates = True
 
-            normal_items = [model.item(i) for i in range(2, model.rowCount())]  # skip dummy and toggle
+            normal_items = [model.item(i) for i in range(2, model.rowCount())]
 
             if item == toggle_item:
                 all_checked = all(i.checkState() == Qt.Checked for i in normal_items)
-                if all_checked:
-                    for i in normal_items:
-                        i.setCheckState(Qt.Unchecked)
-                    toggle_item.setCheckState(Qt.Unchecked)
-                else:
-                    for i in normal_items:
-                        i.setCheckState(Qt.Checked)
-                    toggle_item.setCheckState(Qt.Checked)
-
-            elif item == dummy_item:
-                pass
-
-            else:
-                # When normal items change, update toggle item
+                target_state = Qt.Unchecked if all_checked else Qt.Checked
+                for i in normal_items:
+                    i.setCheckState(target_state)
+                toggle_item.setCheckState(target_state)
+            elif item != dummy_item:
                 all_checked = all(i.checkState() == Qt.Checked for i in normal_items)
                 toggle_item.setCheckState(Qt.Checked if all_checked else Qt.Unchecked)
-                
+
             self._updating_checkstates = False
             
-            for param_name, info in self.widgets.items():
-                if info["widget"] == combo:
-                    self.update_dropdown_label(param_name)
-                    break
+            if param_name:
+                self.update_dropdown_label(param_name)
+                self.check_if_changed(param_name)
 
         model.itemChanged.connect(on_item_changed)
-
-        combo.setInsertPolicy(QComboBox.NoInsert)
         return combo
 
-    def show_help_popup(self, text):
+    def show_help_popup(self, param_name, text):
         msg = QMessageBox(self)
-        msg.setWindowTitle(f"Parameter Info - {APP_NAME.upper()}")
+        msg.setWindowTitle(f"Parameter Info - {param_name} - {APP_NAME.upper()}")
         msg.setText(text)
         msg.exec()
 
-
     def get_param_values(self):
         values = {}
         for name, info in self.widgets.items():
             widget = info["widget"]
             expected_type = info["type"]
+            type_str = str(expected_type).lower()
 
-            if name == "SHORT_CHANNEL_REGRESSION":
-                # If the widget is disabled (greyed out), force False
-                if not widget.isEnabled():
-                    values[name] = False
-                    continue
-
-            if expected_type == bool:
+            if expected_type == bool or "bool" in type_str:
                 values[name] = widget.currentText() == "True"
-            elif expected_type == list:
+            elif expected_type == list or "list" in type_str:
                 if isinstance(widget, FullClickComboBox):
-                    values[name] = [x.strip() for x in widget.lineEdit().text().split(",") if x.strip()]
+                    values[name] = [x.strip() for x in widget.lineEdit().text().split(",") if x.strip() and x.strip() != "<None Selected>"]
                 elif isinstance(widget, QComboBox):
                     values[name] = widget.currentText()
-            elif expected_type == range:
-                if isinstance(widget, QSpinBox):
-                    # Convert the integer N into range(N)
-                    values[name] = range(widget.value())
-                else:
-                    values[name] = range(15) # Fallback
+            elif expected_type == range or "range" in type_str:
+                values[name] = range(widget.value()) if isinstance(widget, QSpinBox) else range(15)
             elif expected_type == "json_file":
                 values[name] = widget.text()
             else:
                 raw_text = widget.text()
                 try:
-                    if expected_type == int:
+                    if expected_type == int or "int" in type_str:
                         values[name] = int(raw_text)
-                    elif expected_type == float:
+                    elif expected_type == float or "float" in type_str:
                         values[name] = float(raw_text)
-                    elif expected_type == str:
-                        values[name] = raw_text
                     else:
-                        values[name] = raw_text  # Fallback
-                except Exception as e:
-                    raise ValueError(f"Invalid value for {name}: {raw_text}") from e
+                        values[name] = raw_text
+                except Exception:
+                    values[name] = raw_text
 
         return values
-    
-    def update_dropdown_items(self, param_name, new_items):
-        """
-        Updates the items in a multi-select dropdown parameter field.
 
-        Args:
-            param_name (str): The parameter name (must match one in self.widgets).
-            new_items (list): The new items to populate in the dropdown.
-        """
-        widget_info = self.widgets.get(param_name)
-        #print("[ParamSection] Current widget keys:", list(self.widgets.keys()))
-
-        if not widget_info:
-            print(f"[ParamSection] No widget found for param '{param_name}'")
-            return
-
-        widget = widget_info["widget"]
-        if not isinstance(widget, FullClickComboBox):
-            print(f"[ParamSection] Widget for param '{param_name}' is not a FullClickComboBox")
-            return
-
-        # Replace the model on the existing widget
-        new_model = QStandardItemModel()
-
-        dummy_item = QStandardItem("<None Selected>")
-        dummy_item.setFlags(Qt.ItemIsEnabled)
-        new_model.appendRow(dummy_item)
-
-        toggle_item = QStandardItem("Toggle Select All")
-        toggle_item.setFlags(Qt.ItemIsUserCheckable | Qt.ItemIsEnabled)
-        toggle_item.setData(Qt.Unchecked, Qt.CheckStateRole)
-        new_model.appendRow(toggle_item)
-
-        for item_text in new_items:
-            item = QStandardItem(item_text)
-            item.setFlags(Qt.ItemIsUserCheckable | Qt.ItemIsEnabled)
-            item.setData(Qt.Unchecked, Qt.CheckStateRole)
-            new_model.appendRow(item)
-
-        widget.setModel(new_model)
-        widget.setView(QListView())  # Reset view to refresh properly
-
-        def on_view_clicked(index):
-            item = new_model.itemFromIndex(index)
-            if item.isCheckable():
-                new_state = Qt.Checked if item.checkState() == Qt.Unchecked else Qt.Unchecked
-                item.setCheckState(new_state)
-
-        widget.view().pressed.connect(on_view_clicked)
-
-        def on_item_changed(item):
-            if getattr(self, "_updating_checkstates", False):
-                return
-            self._updating_checkstates = True
-
-            normal_items = [new_model.item(i) for i in range(2, new_model.rowCount())]
-            if item == toggle_item:
-                all_checked = all(i.checkState() == Qt.Checked for i in normal_items)
-                for i in normal_items:
-                    i.setCheckState(Qt.Unchecked if all_checked else Qt.Checked)
-                toggle_item.setCheckState(Qt.Unchecked if all_checked else Qt.Checked)
-            else:
-                all_checked = all(i.checkState() == Qt.Checked for i in normal_items)
-                toggle_item.setCheckState(Qt.Checked if all_checked else Qt.Unchecked)
-
-            self._updating_checkstates = False
-            
-            for param_name, info in self.widgets.items():
-                if info["widget"] == widget:
-                    self.update_dropdown_label(param_name)
-                    break
-
-        new_model.itemChanged.connect(on_item_changed)
-        widget.lineEdit().setText("<None Selected>")
-        
-        
     def _get_checked_items(self, combo):
         checked = []
         model = combo.model()
@@ -798,20 +801,14 @@ class ParamSection(QWidget):
     def update_dropdown_label(self, param_name):
         widget_info = self.widgets.get(param_name)
         if not widget_info:
-            print(f"[ParamSection] No widget found for param '{param_name}'")
             return
 
         widget = widget_info["widget"]
         if not isinstance(widget, FullClickComboBox):
-            print(f"[ParamSection] Widget for param '{param_name}' is not a FullClickComboBox")
             return
 
         selected = self._get_checked_items(widget)
-        if not selected:
-            widget.lineEdit().setText("<None Selected>")
-        else:
-            # You can customize how you display selected items here:
-            widget.lineEdit().setText(", ".join(selected))
+        widget.lineEdit().setText(", ".join(selected) if selected else "<None Selected>")
         
     # def update_annotation_dropdown_from_loaded_files(self, bubble_widgets, button1):
     #     file_paths = [bubble.file_path for bubble in bubble_widgets.values()]
@@ -860,6 +857,135 @@ class ParamSection(QWidget):
     #     self.update_dropdown_items("REMOVE_EVENTS", sorted(list(common)))
 
 
+class ProgressBubble(QWidget):
+    """
+    A clickable widget displaying a progress bar made of colored rectangles and a label.
+
+    Args:
+        display_name (str): Text to display above the progress bar.
+        file_path (str): Associated file path stored with the bubble.
+
+    """
+        
+    clicked = Signal(object)
+    rightClicked = Signal(object, QPoint)
+    
+    def __init__(self, display_name, file_path):
+        super().__init__()
+
+        self.layout = QVBoxLayout()
+        self.label = QLabel(display_name)
+        self.loading_timer = QTimer(self)
+        self.loading_timer.timeout.connect(self._rotate_spinner)
+        self.spinner_frames = ["◐", "◓", "◑", "◒"] #cute
+        self.spinner_idx = 0
+        self.is_loading = False
+        self.base_text = display_name
+        self.status_icon = ""
+        self.suffix_text = ""
+        self.label.setAlignment(Qt.AlignmentFlag.AlignCenter)
+        self.label.setStyleSheet("""
+            QLabel {
+                border: 1px solid #888;
+                border-radius: 10px;
+                padding: 8px 12px;
+                background-color: #e0f0ff;
+            }
+        """)
+
+        self.progress_layout = QHBoxLayout()
+
+        self.rects = []
+        for i in range(28):
+            rect = QFrame()
+            rect.setFixedSize(10, 18)
+            rect.setStyleSheet("background-color: white; border: 1px solid gray;")
+            stage_name = PIPELINE_STAGES[i]
+            rect.setToolTip(f"Stage {i + 1}: {stage_name}")
+            self.progress_layout.addWidget(rect)
+            self.rects.append(rect)
+
+        self.layout.addWidget(self.label)
+        self.layout.addLayout(self.progress_layout)
+        self.setLayout(self.layout)
+
+        # Store the file path
+        self.file_path = os.path.normpath(file_path)
+
+        self.current_step = 0
+
+        # Make the bubble appear to the user as clickable
+        self.setCursor(Qt.CursorShape.PointingHandCursor)
+
+        # Resize policy to make bubbles responsive
+        self.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Minimum)
+
+
+    def _update_label_text(self):
+        """Combines base text, green checkmark/spinner, and metadata into one display label."""
+        text = self.base_text
+        if self.status_icon:
+            text += f" {self.status_icon}"
+        if self.suffix_text:
+            text += f" ({self.suffix_text})"
+        self.label.setText(text)
+
+
+    def set_loading_state(self, loading=True):
+        self.is_loading = loading
+        if loading:
+            self.loading_timer.start(150) # Rotate every 150ms
+        else:
+            self.loading_timer.stop()
+            # Transition to a green checkmark
+            self.status_icon = "<span style='color: green;'>✔</span>"
+            self._update_label_text()
+
+
+    def setSuffixText(self, suffix):
+        """Updates the metadata text portion without destroying the checkmark."""
+        self.suffix_text = suffix if suffix else ""
+        self._update_label_text()
+
+
+    def update_progress(self, step_index, active=True):
+        self.current_step = step_index
+        for i, rect in enumerate(self.rects):
+            if i < step_index:
+                rect.setStyleSheet("background-color: green; border: 1px solid gray;")
+            elif i == step_index:
+                color = "yellow" if active else "white"
+                rect.setStyleSheet(f"background-color: {color}; border: 1px solid gray;")
+            else:
+                rect.setStyleSheet("background-color: white; border: 1px solid gray;")
+
+                
+    def mark_cancelled(self):
+        for i, rect in enumerate(self.rects):
+            if i < self.current_step:
+                rect.setStyleSheet("background-color: green; border: 1px solid gray;")
+            elif i == self.current_step:
+                rect.setStyleSheet("background-color: red; border: 1px solid gray;")
+            else:
+                rect.setStyleSheet("background-color: white; border: 1px solid gray;")
+
+
+    def mousePressEvent(self, event):
+        if event.button() == Qt.MouseButton.LeftButton:
+            self.clicked.emit(self)
+        elif event.button() == Qt.MouseButton.RightButton:
+            self.rightClicked.emit(self, event.globalPosition().toPoint())
+        super().mousePressEvent(event)
+
+
+    def _rotate_spinner(self):
+        frame = self.spinner_frames[self.spinner_idx % len(self.spinner_frames)]
+        # Using HTML in setText allows us to style the spinner specifically
+        self.status_icon = f"<span style='color: #555;'>{frame}</span>"
+        self.spinner_idx += 1
+        self._update_label_text()
+
+
 class FlaresBaseWidget(QWidget):
     def __init__(self, caller):
         super().__init__()

src/shared/shareddata.py

@@ -13,7 +13,7 @@ import sys
 import platform
 
 
-CURRENT_VERSION = "1.5.2"
+CURRENT_VERSION = "1.6.0"
 APP_NAME = "flares"
 APP_NAME_EXPANDED = "fNIRS Lightweight Analysis, Research, & Evaluation Suite"
 API_URL = f"https://git.research.dezeeuw.ca/api/v1/repos/tyler/{APP_NAME}/releases"
@@ -55,6 +55,20 @@ PIPELINE_STAGES = [
 ]
 
 
+DATA_SCHEMA = [
+    {"key": "raw_haemo_dict",        "help": "Dict[file_path, MNE RawArray]: Haemodynamic raw data"},
+    {"key": "epochs_dict",           "help": "Dict[file_path, MNE Epochs]: Time-locked epoch data"},
+    {"key": "cha_dict",              "help": "Dict[file_path, DataFrame]: Channel analysis results"},
+    {"key": "df_ind_dict",           "help": "Dict[file_path, DataFrame]: Individual-level data/ROI results"},
+    {"key": "design_matrix_dict",    "help": "Dict[file_path, DataFrame]: GLM design matrices"},
+    {"key": "config_dict",           "help": "Dict[file_path, dict]: Processing configuration parameters"},
+    {"key": "fig_bytes_dict",        "help": "Dict[file_path, dict]: Serialized figure data"},
+    {"key": "contrast_results_dict", "help": "Dict[file_path, dict]: Calculated contrast statistical results"},
+    {"key": "roi_channel_map_dict",  "help": "Dict[file_path, dict]: Calculated contrast statistical results"},
+    {"key": "valid_dict",            "help": "Dict[file_path, bool]: Boolean validity status per file"}
+]
+
+
 def resource_path(relative_path: str) -> str:
     """
     Get absolute path to resource regardless of running directly or packaged using PyInstaller