Cross-modality synthesis from CT to MRI provides MRI-level soft-tissue detail without additional scans, yet existing approaches face three major limitations: the substantial modality gap between CT and MRI, slice-level quality variation without explicit estimation, and underutilization of 3D anatomical continuity. While prior work has primarily focused on MRI-to-CT synthesis, CT-to-MRI generation remains challenging. We propose LGN (Learning from Good Neighbors), a framework for synthesizing MRI from CT, consisting of three components: (1) a Latent Diffusion Model (LDM) for draft MRI generation, (2) a slice-level Quality Predictor trained via LPIPS-based teacher–student learning, and (3) a 3D context-aware Cross-Attention Refiner that selectively improves suboptimal slices using high-quality neighboring slices. Experiments on 398 patients demonstrate consistent improvements in LPIPS, PSNR, and MAE with reduced artifacts and enhanced anatomical consistency, highlighting the effectiveness of quality-guided cross-slice refinement for CT-to-MRI synthesis.
Overview of the proposed LGN for CT-to-MRI synthesis. (a) CT-conditioned LDM for draft MRI generation. (b) A teacher--student Quality Predictor estimates quality. (c) A Cross-Attention Refiner leverages good neighbors to refine the draft.
Examples of CT and MRI images in the refinement process. Red boxes highlight artifact regions. $Q$ denotes the quality score predicted by the Quality Predictor.
Table 1 shows that adding the Refiner improves all metrics over LDM only even without neighbors (using only the target slice and CT), and incorporating neighboring slices in Full (Ours) further reduces LPIPS from 0.066 to 0.061 for b=0 and from 0.070 to 0.068 for b=1000, demonstrating the benefit of anatomical context from high-quality neighbors. In contrast, LDM+Refiner (w/o CT) underperforms Full, indicating that CT conditioning and neighboring-slice context are complementary.
Table 2 further shows that LGN performs best with three neighboring slices. Using too few neighbors (0 or 1) provides limited contextual benefit, whereas using too many neighbors (5 or all) can dilute useful context and degrade refinement performance.
Table 3 provides quantitative comparison against existing state-of-the-art methods. LGN achieves the best LPIPS and MAE (lower is better) and the highest PSNR for both b=0 and b=1000.
Figure 3 shows qualitative results for each method. Note that our goal is anatomically faithful CT-to-MRI synthesis, not lesion detection, and we do not quantitatively evaluate lesion detectability.
