Skip to content

EyeWorld: A Generative World Model of Ocular State and Dynamics

🕒 Published (v1): 2026-03-14 17:19 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

EyeWorld is a generative world model that treats the eye as a partially observed dynamical system, learning a single observation-stable latent ocular state shared across seven imaging modalities (CFP, OCT, FFA, ICGA, MRI, AF, UWF). It unifies fine-grained segmentation, cross-modality translation, image enhancement, and longitudinal disease forecasting within one framework initialized from OmniGen2. By incorporating temporal supervision, EyeWorld can simulate clinically meaningful disease progression rather than merely recognizing static patterns.

Problem

Most ophthalmic AI systems are static, single-task, and single-modality, failing to generalize under modality shifts, acquisition variability, or across follow-up time points. Existing medical foundation models remain representation-centric — they encode images but do not model the eye as a dynamical system with evolving state — making them poorly suited for prognosis-oriented reasoning and robust multimodal integration.

Method

EyeWorld is initialized from the OmniGen2 backbone and restructured into a domain-specific world model. A dual-path language-vision encoder tokenizes multimodal images and free-form text prompts into a shared latent embedding. An "Ophthalmic World Simulator" module operates on this latent state to support all clinical objectives. For temporal tasks, the model learns explicit time-conditioned latent state transitions from longitudinal follow-up data (pre-op, post-op, follow-up visits), enabling forward simulation rather than static input-output mapping. A unified generative projection head decodes the latent state into segmentation masks, cross-modality synthesized images, or future-state predictions. Training uses 85,638 image-mask pairs (54 structure/lesion types), 31,179 cross-modality pairs, and 16,645 longitudinal examinations across 108,672 total images.

Key Contributions

  • A generative world-model framework for ophthalmology that learns a single observation-stable latent ocular state across 7 imaging modalities
  • Time-conditioned latent state transitions enabling longitudinal disease progression forecasting (AMD, macular hole) over clinically relevant intervals
  • Unified single-framework handling of segmentation, detection, cross-modality translation, quality enhancement, and longitudinal prediction — no task-specific architectural branches
  • Counterfactual progression synthesis: selective perturbation of pathological factors while preserving unrelated anatomy, enabling "what-if" progression simulation
  • Exemplar-conditioned task control: model infers the transformation from a reference input-output pair and applies it to a new query without retraining

Results

  • Segmentation (60 tasks, 12,845 pairs): EyeWorld mean Dice 0.77 vs. best baseline Qwen-Image 0.69; anatomical structures Dice 0.84 vs. 0.76; lesions Dice 0.66 vs. 0.58
  • CFP anatomy: Dice 0.858, outperforming OmniGen2 by 8.72%, Step1X-Edit by 31.82%
  • CFP lesions: Dice 0.71, exceeding Qwen-Image by 5% (all P < 0.05)
  • Ambiguous boundary segmentation (optic disc/cup): Dice 0.865, mIoU 0.764 vs. OmniGen2 0.778 (next best on Dice)
  • Detection (14 tasks): EyeWorld highest tasks with IoU > 0.7 (4 tasks) vs. OmniGen2 (3), ChronoEdit/Qwen-Image (1 each), Step1X-Edit (0)
  • Cross-modality translation (18 directions): Best LPIPS/PSNR/SSIM distributions; CFP→FFA IS improvement 4.05%, ICGA→CFP 8.12% over best baseline
  • Image enhancement LPIPS: 0.142 (degradation), 0.170 (SR), 0.215 (inpainting), 0.301 (outpainting); combined degradation outperforms Step1X-Edit by 0.208 LPIPS
  • Longitudinal prediction (AMD + macular hole OCT): Mean LPIPS 0.371, SSIM 0.651, PSNR 21.462; exceeds strongest baseline by 2.3% LPIPS, 6.1% SSIM, 2.278 dB PSNR
  • Quality robustness: Degraded-input Dice 0.722 vs. 0.623 (OmniGen2, next best); enhance-then-analyze improves EyeWorld (0.722→0.731) while hurting all baselines

Limitations

  • Class imbalance acknowledged for rare lesion targets (discussion truncated in provided text)
  • Evaluated exclusively on ophthalmic data; generalizability to other medical imaging domains not assessed
  • Baselines (Step1X-Edit, ChronoEdit, OmniGen2, Qwen-Image) are general-purpose editors fine-tuned on the same data — comparison against dedicated ophthalmic-specific models is absent
  • Temporal evaluation limited to OCT (AMD and macular hole); longitudinal modeling across other modalities (CFP, angiography) not demonstrated
  • Counterfactual synthesis is qualitatively assessed via difference maps; no clinical validation of generated progression scenarios

Relevance to Foundation Models in Medicine

EyeWorld directly challenges the dominant paradigm in medical foundation models — static representation learning — by reframing medical imaging as dynamical system inference, a shift with broad implications beyond ophthalmology. It demonstrates that a single generalist latent state can simultaneously support discriminative (segmentation, detection) and generative (synthesis, enhancement, forecasting) objectives without task-specific heads, a key desideratum for practical clinical deployment. The world-model framing — learning observation-stable state representations plus time-conditioned transitions — offers a transferable architectural blueprint for other imaging-intensive specialties (radiology, cardiology, pathology) where longitudinal reasoning and multimodal integration are similarly central. The exemplar-conditioned task interface points toward few-shot clinical AI that can adapt to new tasks without retraining, which is a critical capability gap in current medical foundation models.