4KAgent: Agentic Any Image to 4K Super-Resolution¶

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

4KAgent is a training-free multi-agent framework that upscales any image to 4K resolution by combining a VLM-based Perception Agent for degradation reasoning and restoration planning with a Restoration Agent that executes a quality-driven mixture-of-experts (Q-MoE) policy under an execution–reflection–rollback loop. It generalizes across 11 task categories—natural images, portraits, AIGC, satellite, microscopy, and medical modalities—without domain-specific retraining. Evaluated on 26 benchmarks, it achieves state-of-the-art on perceptual metrics across most tasks.

Problem¶

Existing super-resolution methods are limited to narrow degradation ranges (typically ≤4× upscaling), generalize poorly to out-of-distribution domains (scientific, medical, AIGC), and cannot flexibly adapt to heterogeneous user workflows (e.g., fidelity vs. perceptual quality preference). No prior system handles arbitrary input types, extreme upscaling factors (16×), and diverse domain requirements within a single, retraining-free framework.

Method¶

4KAgent decomposes super-resolution into three modular components:

Perception Agent: Runs four no-reference IQA tools (CLIPIQA, TOPIQ, MUSIQ, NIQE) on the input, then feeds the scores plus the raw image to a VLM (default: Llama-Vision) to predict degradation types, generate image captions, and produce a restoration plan. It also auto-computes the scale factor s = min{ŝ ∈ {2,4,8,16} | max(H,W)·ŝ ≥ 4000}.
Restoration Agent: Executes the plan step-by-step using a toolbox of 9 restoration operators. At each step, all tools run in parallel producing N candidate outputs (execution); Q-MoE selects the best via a composite score combining HPSv2 human preference and weighted no-reference IQA metrics (reflection). If the winner's score falls below threshold η, a rollback discards it and reassigns the step to a different operator.
Face Restoration Pipeline: Triggered post-SR; detects faces, applies N face restoration experts per face, and selects the best via a score combining ArcFace cosine identity preservation and CLIB-FIQA quality.

A Profile Module exposes seven user-configurable parameters (VLM choice, scale factor, restore tasks, face restore toggle, fidelity vs. perception preference, etc.) enabling zero-retraining adaptation to new tasks.

Key Contributions¶

First agentic framework for universal any-image-to-4K super-resolution (up to 16×, iteratively higher) without retraining.
Multi-agent system design: VLM-driven Perception Agent + execution–reflection–rollback Restoration Agent.
Quality-Driven Mixture-of-Experts (Q-MoE) policy combining HPSv2 + no-reference IQA for step-level selection.
Dedicated face restoration pipeline with identity-aware Q-MoE scoring (ArcFace + CLIB-FIQA).
Profile Module enabling zero-shot task adaptation via configuration.
DIV4K-50 dataset: 50 images with 4096×4096 HQ ground truth and 256×256 degraded LQ inputs for evaluating 16× joint restoration+SR.
MUSIQ-P patch-based perceptual metric for evaluating local quality in 4K images.

Results¶

Classical SR (B100, 4×): 4KAgent (ExpSR-s4-F) matches HAT-L at PSNR 28.09 / SSIM 0.7540; perceptual profile (GenSR-s4-P) achieves NIQE 3.93 and MUSIQ 69.44 vs. AgenticIR's NIQE 4.08 / MUSIQ 68.36.
Real-World SR (RealSR, 4×): GenSR-s4-P achieves best NIQE 4.78 and MUSIQ 71.77 among all methods; outperforms AgenticIR on every metric.
Multiple-Degradation Restoration (MiO100 Group C): 4KAgent achieves PSNR 19.77, MUSIQ 55.56, CLIPIQA 0.5233, MANIQA 0.3545—state-of-the-art on all metrics except LPIPS.
16× SR (RealSRSet): Qualitatively superior to HAT-L (4×→4×) and DiffBIR; quantitative results deferred to appendix.
AIGC SR (GenAIBench-4K): Consistently improves NIQE and CLIPIQA over all base generators (SANA, GPT-4o, FLUX, PixArt-Σ, SD3-Medium); e.g., FLUX NIQE 6.18→2.98.
Evaluated across 11 task categories, 26 benchmarks total; sets new SoTA on perceptual metrics in scientific domains (satellite, fluorescence microscopy, X-ray, ultrasound, fundoscopy, pathology).

Limitations¶

Perceptual-fidelity tradeoff not fully resolved: GenSR profiles improve NIQE/MUSIQ but degrade PSNR/SSIM/LPIPS vs. fidelity-focused or discriminative baselines.
Inference is slow due to running all tools at each step for Q-MoE selection; efficiency not addressed in current design.
Robustness and safety risks (privacy leakage, harmful content generation from generative priors) are acknowledged but unresolved.
Profile tuning requires manual selection of configuration; automation of profile recommendation is not demonstrated.
16× and joint-restoration quantitative results are in the appendix (not fully presented in main paper text).
DIV4K-50 dataset is synthetically degraded; gap to truly in-the-wild extreme degradations may exist.

Relevance to Vision-Language Models¶

The Perception Agent is the central VLM application: a vision-language model (Llama-Vision by default, swappable via profile) interprets IQA metric outputs alongside the raw image to diagnose degradation types and generate a structured restoration plan in natural language, acting as the reasoning backbone of the entire pipeline. This is a concrete instantiation of VLMs as low-level vision planners rather than high-level semantic reasoners, extending their role into image quality-aware agentic workflows. The work demonstrates that VLMs can bridge perceptual quality signals and sequential decision-making in restoration without task-specific fine-tuning, which is relevant to researchers studying grounded VLM reasoning and tool-use agents. It also provides a reproducible benchmark (DIV4K-50, 26 evaluation datasets) against which future VLM-based perception and planning modules can be measured.