Project
Plenoptima
In medical image analysis (MI), achieving high-resolution detail is crucial for accurate diagnoses, but remains restricted by extended acquisition times and the need for high frame rates. Existing deep learning-based superresolution (SR) techniques, which often employ simplified degradation models (e.g., Additive White Gaussian Noise (AWGN)), fail to capture the complex, modality-dependent noise characteristics encountered in clinical settings. To address these limitations, we introduce a Neural Parametric Steered Mixture of Experts (N-SMoE) framework, trained within a Generative Adversarial Network (GAN) paradigm, and incorporating an advanced Stochastic Degradation Model (SDM). The SDM applies diverse levels of blurring and noise to downsampled images, thus simulating realistic clinical conditions and mitigating training instability. In contrast to conventional bilinear or bicubic interpolation approaches, the N-SMoE encoder leverages a lightweight, multilayer Laplacian resizer with bandpass filtering capabilities, while a multi-head attention module isolates high-frequency structural patterns to estimate Gaussian primitives for the decoder. The SMoE decoder, unlike conventional deep learning (DL) black-box models, utilizes gating functions and two-dimensional kernels to produce an edge-aware representation capable of modeling both continuous and smooth transitions, which makes it an excellent tool for SR-related restoration tasks. The resulting proposed neural parametric autoregressive framework enhances interpretability and surpasses state-of-the-art (SotA) methods in terms of fidelity, perceptual quality, and subjective evaluations across multiple medical imaging modalities.
Project Details
Project term
April 4, 2024–December 31, 2024
Affiliations
Technical University of Berlin
Institute
Communication Systems Group
Principal Investigator
Methods
We proposed Neural parametric Steered Mixture of Experts regression framework which uses Generative Adversarial Network training paradigm for Advanced imaging modalities, including magnetic resonance imaging (MRI), computed tomography (CT), optical coherence tomography (OCT), and ultrasound.
Results
Our proposed approach achieved state of the art performance, and we submitted it to Elsevier Signal Processing Journal.
Discussion
The proposed method yields promising and convincing results; however, it requires a significant amount of computational resources. Our current focus is on developing an environmentally friendly approach.
Additional Project Information
DFG classification: 408 Electrical Engineering and Information Technology
Software: PyTorch, CUDA
Cluster: CLAIX
Publications
Thesis
Aytaç Özkan, neural Parametric Steered Mixture of Experts for Inverse Prblems, PhD dissertation, will submit March 2025.