VaseMuseum: Digital Intelligent Museum for Ancient Greek Pottery¶

🕒 Published (v1): 2026-07-07 15:11 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

VaseMuseum is a modular, training-free multimodal agent framework for intelligent digital museums of ancient Greek pottery. It wraps a frozen VLM backbone with source-level and response-level reliability controls around an iterative DeepResearch-style retrieval loop to reduce hallucinations and produce calibrated uncertainty when heritage evidence is weak or contested. A GRPO-style reranker selects among multiple stochastic rollouts without gradient updates.

Problem¶

General-purpose VLMs deployed in cultural-heritage settings face two compounding failures: (1) open-ended queries require external scholarly knowledge, but web retrieval injects low-quality, inaccessible, or unverifiable citations into the context; (2) when evidence is partial, disputed, or absent, VLMs emit fluent but unsupported answers instead of expressing calibrated uncertainty. Existing approaches either rely on closed-set retraining or leave retrieval output uncontrolled.

Method¶

VaseAgent processes a query $q$ and exhibit image $I$ through four stages:

Iterative retrieval. A DeepResearch-style tool loop decomposes $q$ into search intents, issues web/museum-source queries, and refines iteratively based on intermediate reasoning (Algorithm 5).

Source control. Each retrieved hit $h$ is scored by $s_\text{val}(h) = 0.55\,a(h) + 0.45\,c(h)$, where $a(h)$ is a URL accessibility proxy and $c(h)$ is text sufficiency; hits below $s_\text{val} < 0.40$ or $a(h) < 0.50$ are dropped. Surviving hits are ranked by a pre-diversity score $\varphi_\text{pre}(h)$ combining query relevance, source quality, and accessibility, then greedily selected up to $N_\text{src}=5$ via Maximal Marginal Relevance (MMR) with $\lambda_\text{MMR}=0.72$.

Response control. The draft answer is decomposed into coarse claim units; lexical overlap between each claim $c_i$ and evidence $e_j$ triggers support at threshold $\tau_\text{sup}=0.08$. A scalar confidence score combines coverage, consistency, multi-source backing, and conflict penalties: $$\psi = 0.40\,c_\text{cov} + 0.30\,\kappa_\text{cons} + 0.20\,\omega_\text{multi} + 0.10\,m_E - \min(0.35,\, 0.12\,n_\text{conf})$$ If $\psi < \tau_\text{conf} = 0.52$, the agent prepends an evidence-bounded uncertainty preamble.

Training-free GRPO-style selection. When inference budget permits, $K=4$ independent controlled trajectories are sampled and reranked by an aggregated reliability score $R(\cdot)$ covering link validity, claim–evidence support, neutrality, and conflict awareness—no VLM weight updates occur.

Evaluation uses a 300-instance set (100 visual-only / 100 visual+knowledge / 100 ambiguous) derived from VaseVQA-3D, enriched with LIMC iconographic database links.

Key Contributions¶

VaseMuseum framework: interactive virtual museum coupling 2D/3D artifact exploration with an open-domain multimodal reasoning agent for ancient Greek pottery.
VaseAgent: inference-time agent integrating iterative DeepResearch-style retrieval with deterministic source-control filtering (validity, diversity, MMR) for reliable evidence grounding.
Response-level reliability control: lexical claim–evidence auditing with a composited confidence score $\psi$ and threshold-triggered uncertainty-bounded responses.
Training-free GRPO-style selector: multi-rollout reranking by reliability verifiers, applicable without modifying the VLM backbone.
Evaluation protocol: three-way task taxonomy (visual-only, visual+knowledge, ambiguous) with link-validity and neutrality metrics suited to cultural-heritage open-domain QA.

Results¶

The paper text is truncated before the quantitative tables; the following are the reported directional findings: - VaseMuseum improves citation link validity over search-enabled VLM baselines. - Hallucination rate is reduced on visual+knowledge (knowledge-intensive) queries versus baselines. - Neutral/appropriately hedged responses increase on ambiguous questions versus baselines. - Qualitative examples show reduced fabricated or over-specific explanations relative to unconstrained retrieval-augmented VLMs.

(No absolute accuracy numbers are present in the provided text.)

Limitations¶

Domain is restricted to ancient Greek pottery; generalization to other heritage domains is asserted but not validated.
Source-control thresholds ($\tau_\text{keep}=0.22$, $\tau_\text{conf}=0.52$, $\tau_\text{sup}=0.08$, $N_\text{src}=5$) are fixed heuristics without ablation sensitivity analysis in the provided text.
Lexical-overlap claim–evidence matching is shallow and may miss paraphrase or entailment relationships.
GRPO-style selection multiplies inference cost by $K=4\times$; not viable under strict latency constraints.
Evaluation set is small (300 instances) and pottery-specific; results may not transfer to other collection types or query distributions.
Claim decomposition relies on rule-based sentence/clause splitting, which can fail on complex or multi-part heritage questions.

Relevance to Vision-Language Models¶

VaseMuseum directly addresses a known failure mode of VLMs—confident hallucination under distributional shift—by interposing deterministic, training-free reliability controls at inference time rather than relying on larger models or domain fine-tuning. The source-control and response-control layers constitute a modular "wrapper" applicable in principle to any frozen VLM backbone, making the approach relevant to researchers studying RAG-augmented VLMs, uncertainty calibration, and grounded multimodal reasoning. The training-free GRPO-style reranker connects to the broader thread of using RL-inspired preference scoring over inference-time samples without gradient updates, a direction gaining traction in VLM alignment and test-time compute scaling. For VLM practitioners, this work provides a concrete case study of how domain-specific deployment constraints (citation fidelity, uncertainty expression, 3D artifacts) shape architecture decisions beyond standard benchmarks.