Multi-Agent Multimodal Large Language Model Framework for Automated Interpretation of Fuel Efficiency Analytics in Public Transportation

🕒 Published (v1): 2025-11-17 15:14 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

A three-agent pipeline — narration agent, LLM-as-a-judge, and optional human-in-the-loop — automates end-to-end conversion of multimodal analytics artifacts (charts, tables, numerical outputs) into stakeholder-ready reports. Validated on 4006 real bus trips from Northern Jutland, Denmark, using Gaussian Mixture Model clustering. GPT-4.1 mini with Chain-of-Thought prompting achieves 97.3% narrative accuracy across a five-model, three-paradigm comparison.

Problem

Industrial analytics pipelines produce fragmented outputs (figures, tables, model summaries) that require costly manual synthesis before they are usable for decision-making. Existing LLM-based narration approaches focus on isolated subtasks (chart captioning, image-text reasoning) and lack end-to-end integration with validation, iterative refinement, and stakeholder-oriented reporting — particularly in energy-critical domains like transportation.

Method

The framework defines a reusable block unit consisting of: (1) a data narration agent that ingests multimodal inputs and produces a structured textual narrative via zero-shot LLM inference; (2) an LLM-as-a-judge agent that scores the narrative on four dimensions — clarity, relevance, insightfulness, contextualization — each on a 0–4 scale, returning feedback that triggers iterative refinement until a target score or cycle limit is reached (temperature = 0.1 for determinism); (3) an optional human-in-the-loop evaluator for domain-sensitive cases.

Blocks are composed into a four-stage workflow: (i) raw data description → (ii) data modeling → (iii) post hoc analytics → (iv) integration and narration. Contextual knowledge accumulates cumulatively as Background → Background+ → Background++, carrying both prior background and generated narratives into each subsequent stage. Within each stage, blocks run in parallel over their respective inputs. The final reporting agent synthesizes all narratives into a structured document (executive summary, methodology, findings, recommendations) without altering factual content.

The evaluation compares five LLMs (GPT-4.1, GPT-4.1 mini, o4-mini, Claude 3.5 Haiku, Gemini 2.5 Flash) across three prompting strategies (zero-shot, few-shot, Chain-of-Thought) on the fuel efficiency case study.

Key Contributions

• Three-agent block abstraction with automated iterative refinement loop (narration ↔ judge) and deterministic judge evaluation (temperature = 0.1)
• Four-stage cumulative-context workflow enabling transparent, traceable information flow across pipeline stages
• Systematic cross-model, cross-prompting-strategy evaluation identifying optimal configuration (GPT-4.1 mini + CoT)
• Domain-specific deployment in energy informatics: fuel efficiency analytics over 4006 real bus trips using GMM clustering
• Use of distribution figures rather than raw tabular data to mitigate LLM hallucination

Results

• GPT-4.1 mini + Chain-of-Thought achieves 97.3% narrative accuracy, identified as optimal balance of interpretability and computational cost
• Evaluated across 5 LLMs × 3 prompting paradigms (zero-shot, few-shot, CoT); full per-model breakdown not available in the provided excerpt
• Multi-agent orchestration with iterative refinement outperforms single-agent one-shot baselines on factual precision, coherence, and scalability (qualitative claim; quantitative breakdown truncated in provided text)

Limitations

• Post hoc analysis artifacts must be provided externally by data scientists; the system does not autonomously select or suggest analysis methods
• Providing raw tabular data directly to LLMs produces hallucinated content; the framework mitigates this by requiring pre-computed distribution figures, constraining applicability to projects that already produce such visualizations
• Validated on a single domain (bus fuel efficiency, Northern Jutland) and one clustering method (GMM); generalizability to other energy informatics domains is asserted but not demonstrated empirically in the provided text
• LLMs are acknowledged to have fundamental limitations in symbolic and logical reasoning, restricting the framework to descriptive/communicative tasks

Relevance to Harnesses / Meta-Harnesses

This paper is a direct instance of a domain-specific meta-harness: a reusable orchestration layer that composes specialized LLM agents into a structured, multi-stage pipeline with defined information contracts between stages. The block abstraction — encapsulating narration + judge + optional human oversight into a composable unit — mirrors the module/stage primitives found in general meta-harness architectures. The cumulative context propagation scheme (Background → Background++) is a concrete example of state threading across pipeline stages, a key harness design pattern. The comparative evaluation of LLM backends and prompting strategies as swappable harness parameters is directly informative for anyone designing configurable, model-agnostic harnesses.