Skip to content

4KAgent: Agentic Any Image to 4K Super-Resolution

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

4KAgent is a multi-agent framework that universally upscales any image to 4K resolution without domain-specific retraining, covering natural photos, portraits, AIGC, and scientific imaging (microscopy, medical, remote sensing). A Perception Agent (VLM + IQA tools) plans a restoration sequence; a Restoration Agent executes it with a Quality-Driven Mixture-of-Experts (Q-MoE) selection and execution–reflection–rollback loop. It achieves state-of-the-art perceptual quality across 26 benchmarks spanning 11 task categories.

Problem

Existing super-resolution models are specialist systems: they generalize poorly across domains, are limited to narrow upscaling ranges (typically ≤4×), and cannot adaptively handle the diverse, compound degradations found in real-world or scientific imagery. There is no unified, training-free agentic system capable of flexible, high-quality restoration across heterogeneous image types and degradation patterns.

Method

4KAgent is a three-component multi-agent pipeline:

  1. Perception Agent: Runs four no-reference IQA tools (CLIPIQA, TOPIQ, MUSIQ, NIQE) on the input, feeds scores plus the raw image into a VLM (default: Llama-3.2-Vision) to predict degradation types and image descriptions, then constructs an ordered restoration plan. Upscaling factor is auto-determined to reach ≥4K.

  2. Restoration Agent: Executes the plan step-by-step using a toolbox of 9 restoration operations (denoising, deblurring, dehazing, deraining, JPEG artifact removal, SR, etc.). At each step, all applicable tools run in parallel producing N candidates. A Q-MoE reflection stage scores each candidate via HPSv2 (human preference, conditioned on the caption) plus a weighted sum of NIQE/MANIQA/MUSIQ/CLIPIQA, and selects the highest scorer. If the selected output's quality score falls below threshold η, a rollback reverts to the prior image and substitutes a different restoration task.

  3. Face Restoration Pipeline: Triggered post-SR; detects and crops faces, applies multiple face restoration tools, selects the best via a composite score combining ArcFace identity preservation (cosine similarity) and CLIB-FIQA, then pastes restored faces back.

A Profile Module exposes seven user-configurable parameters (LLM/VLM choice, scale factor, restoration tasks, face restore toggle, perceptual vs. fidelity preference) enabling zero-retraining adaptation across domains.

Key Contributions

  • First agentic framework for universal any-image-to-4K super-resolution without retraining.
  • Q-MoE policy: parallel execution of all restoration experts per step, quality-driven expert selection combining HPSv2 and no-reference IQA.
  • Execution–reflection–rollback triplet enabling self-correcting iterative restoration.
  • Face Restoration Pipeline with identity-preserving Q-MoE selection via ArcFace + CLIB-FIQA.
  • Profile Module for zero-shot task customization across domains (fidelity vs. perception, scale factor, specific operators).
  • DIV4K-50 dataset: 50 images downsampled from 4096Ă—4096 to 256Ă—256 with compound degradations for 16Ă— SR evaluation.
  • SoTA across 26 benchmarks covering 11 task categories including satellite, microscopy, and medical modalities.

Results

  • Classical SR (B100, 4Ă—): 4KAgent (ExpSR-s4-P) achieves NIQE 3.86, MUSIQ 69.42 vs. AgenticIR's NIQE 4.08, MUSIQ 68.36; fidelity profile (ExpSR-s4-F) PSNR 28.09 vs. AgenticIR 22.51.
  • Real-World SR (RealSR, 4Ă—): 4KAgent (GenSR-s4-P) sets new SoTA on perceptual metrics: NIQE 4.78, MUSIQ 71.77 vs. best prior (SeeSR) MUSIQ 69.37.
  • Multiple-Degradation Restoration (Group C, MiO100): 4KAgent (GenMIR-P) PSNR 19.77, MANIQA 0.3545, CLIPIQA 0.5233, MUSIQ 55.56, surpassing AgenticIR (MUSIQ 48.68) and MAIR (MUSIQ 51.36) on all metrics except LPIPS.
  • AIGC 4K SR (GenAIBench-4K): 4KAgent consistently improves NIQE across all five base generators (e.g., FLUX.1-dev: 6.18→2.98, PixArt-ÎŁ: 4.12→2.76); CLIPIQA gains in most cases.
  • 16Ă— SR: Visual comparison shows 4KAgent produces finer textures than HAT-L and DiffBIR in both 4×→4Ă— and direct 16Ă— settings.

Limitations

  • Rollback and multi-expert execution at each step increase inference latency significantly compared to single-pass models.
  • LPIPS score is not best-in-class on the multiple-degradation benchmark (Group C), indicating possible sharpness/fidelity trade-off.
  • Profile Module requires manual selection of appropriate profiles; the Perception Agent may misdiagnose degradation types for highly unusual inputs.
  • Privacy and harmful content risks acknowledged but not yet mitigated.
  • Evaluation of 16Ă— SR is qualitative only in the main paper; quantitative results deferred to the appendix.
  • Generalizability to video or 3D content is not addressed.

Relevance to Agentic AI / LLM Agents

4KAgent is a direct instantiation of the LLM-as-orchestrator paradigm applied to low-level vision: a VLM plans a multi-step restoration workflow and a separate agent executes it with self-reflection and rollback, demonstrating that agentic architectures can outperform monolithic specialist models even in perception-heavy, non-language domains. The Q-MoE policy—run all experts, score outputs, select the best—is a concrete quality-aware action-selection mechanism transferable to other tool-use agent settings. The rollback-on-quality-failure mechanism extends the standard ReAct/reflection loop with a quantitative termination condition, offering a reusable design pattern. For researchers tracking agentic AI, this work illustrates how to compose VLMs, IQA tools, domain expert models, and configurable profiles into a robust zero-shot generalist agent that rivals fine-tuned specialists.