DroneIQA-VLE: Multi-Task Drone Image Quality Assessment via Vision-Language Ensemble

🕒 Published (v1): 2026-07-01 04:17 UTC · Source: Arxiv · link

Why this paper was selected

VLM ensemble for drone image quality assessment; ICME 2026 challenge solution

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DroneIQA-VLE is a multi-task image quality assessment framework for low-altitude UAV imagery that ensembles SigLIP2 vision encoders and a LoRA-adapted Qwen3.5-9B MLLM to jointly predict global, target-region, and background quality scores. It placed 2nd in the ICME 2026 Drone-IQA Grand Challenge, achieving PLCC/SRCC of 0.9484/0.9420.

Problem

Standard no-reference IQA methods assess only global perceptual quality and fail to model the spatially non-uniform degradations, small target regions, and complex backgrounds characteristic of UAV imagery. No existing method jointly accounts for target-region usability and background interference as auxiliary quality dimensions to improve global quality prediction.

Method

Two complementary pipelines are trained independently and their global-quality predictions arithmetically averaged:

1. SigLIP2 multi-task regression: Two SigLIP2 ViT backbones (ViT-L/16 at 384 px and ViT-SO400M/14 at 378 px) each produce a shared feature vector (1024-d and 1152-d respectively), fed into three independent two-layer FC regression heads (dim → 128 → 1) for global, target, and background quality. Six checkpoints (2 architectures × 3 random-seed splits) are averaged after per-split four-parameter logistic mapping. Training loss is the sum of PLCC losses across all three tasks: \(\mathcal{L} = \sum_{t \in T} \mathcal{L}_{\text{PLCC}}^{(t)}\).

2. Qwen3.5-9B MLLM regression: The model is fine-tuned via LoRA (rank 64, \(\alpha=128\)) applied to all linear modules across the visual encoder, visual aligner, and LLM. Hidden representations replace text generation; a regression head maps them to three quality scores. The training loss combines a fidelity loss \(\mathcal{L}_{\text{Fid}}\) (pairwise ranking consistency via Gaussian CDF \(\Phi\)) and a PLCC-induced correlation loss: \(\mathcal{L} = \lambda_1 \mathcal{L}_{\text{Fid}} + \lambda_2 \mathcal{L}_{\text{PLCC}}\).

Ensemble: \(\hat{Q} = (Q_{\text{SigLIP2}} + Q_{\text{Qwen}})/2\).

Key Contributions

• Multi-task learning framework that uses target and background quality as auxiliary supervision to improve global UAV image quality prediction.
• Cross-architecture ensemble combining spatial quality features (SigLIP2 ViT) with semantic/perceptual reasoning from a LoRA-adapted 9B MLLM.
• Fidelity loss formulation enforcing pairwise rank consistency between predicted and ground-truth quality scores.
• 2nd-place solution on the ICME 2026 Drone-IQA Grand Challenge benchmark (~6,000 UAV images, 18 raters, three quality dimensions).

Results

• DroneIQA-VLE (2nd place): PLCC = 0.9484, SRCC = 0.9420, Score = 0.9452
• 1st place (cmsr): PLCC = 0.9512, SRCC = 0.9450, Score = 0.9481 (gap < 0.3 pp)
• 3rd place (Echo): Score = 0.9363 (DroneIQA-VLE outperforms by ~0.9 pp)
• Test set is held-out; training set = 3,600 images, validation = 1,200 images from VisDrone and UAVDT.

Limitations

• Ensemble of two large models (SigLIP2 ViTs + 9B-parameter MLLM) incurs substantial inference cost; no efficiency analysis is provided.
• The arithmetic averaging ensemble weight is fixed at 0.5 with no learned or tuned weighting reported.
• Generalization beyond the VisDrone/UAVDT distribution is untested; the challenge dataset is the only evaluation domain.
• No ablation is presented isolating the contribution of multi-task auxiliary supervision versus the cross-architecture ensemble.

Relevance to Vision-Language Models

This work demonstrates a practical pattern for adapting MLLMs to regression tasks: bypassing text generation in favor of hidden-state regression heads with LoRA fine-tuning, which is transferable to other VLM-based perceptual scoring tasks. It shows that SigLIP2 (a vision-language encoder) and a generative MLLM (Qwen3.5-9B) capture complementary quality-relevant signals, reinforcing the value of VLM pretraining for downstream perceptual tasks beyond recognition. The use of PLCC and fidelity losses to supervise VLM fine-tuning for ordinal regression is a concrete methodological contribution for the IQA-with-VLMs subfield.