Skip to content

TrajAgent: An LLM-Agent Framework for Trajectory Modeling via Large-and-Small Model Collaboration

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TrajAgent is an LLM-agent framework for automated trajectory modeling that orchestrates large language models with small specialized deep-learning models. It introduces UniEnv—a unified data/model execution environment—and a collaborative learning schema that iteratively co-optimizes data augmentation, model parameters, and agent reasoning. Across four tasks and four real-world datasets, it achieves 2.38%–69.91% gains over task-specific baselines.

Problem

Trajectory modeling (prediction, recovery, classification, generation, representation) suffers from severe task and data heterogeneity: existing specialized models are tied to specific tasks, data formats, and geographic regions and cannot be transferred. Unified multi-task models exist but underperform specialists and are hard to train and configure. There is no automated, extensible framework that can select, configure, and optimize the right model for an arbitrary trajectory task on arbitrary data without expert intervention.

Method

TrajAgent wraps a four-stage agentic workflow (task understanding → task planning → task execution → task summary) over a unified execution substrate called UniEnv:

  • UniEnv provides a unified data interface (check-in / GPS / road-network formats), a model library of 18 methods across 5 task categories with LLM-generated semantic descriptions for each model and dataset, and an external tools pool (Optuna, movingpandas, txyz.ai).
  • Each workflow stage is an independent agent with short- and long-term memory plus a self-reflection (Reflexion-style) module.
  • The Collaborative Learning Schema (CLS) couples high-level agent reasoning with low-level model training in a closed feedback loop. The agent proposes data augmentation operators, parameter configurations, and prompt adjustments; the small model trains, reports metrics back, and the agent updates its memory. Three optimization modes are supported: data augmentation (DA), parameter optimization (PO), and joint optimization (JO = DA then PO); prompt optimization (PRO) applies when the specialized model is itself LLM-based. The stopping criterion is either a performance threshold or a maximum exploration epoch count.

The overall objective is: $\(M' = \arg\min_{M} \mathcal{L}(M, T, q, D, A), \quad T = A(q),\; D = A(q,T),\; M = A(T,D)\)$

Key Contributions

  • TrajAgent: first LLM-agent framework for automated, unified trajectory modeling across heterogeneous tasks and datasets.
  • UniEnv: unified execution environment integrating 18 specialized models, multiple data formats, and external tools behind a single interface, enabling zero-friction extensibility.
  • Collaborative Learning Schema: closed-loop, feedback-driven optimization jointly adapting data augmentation, model hyperparameters, and agent reasoning; includes ablation confirming that score-guided reflection (not pattern matching) drives improvement.
  • Comprehensive evaluation: four real-world datasets (FSQ, BGK, Porto, Tencent/Beijing/Earthquake/Chengdu), ten subtasks, performance gains of 2.38%–69.91% over unoptimized baselines.

Results

  • Next location prediction (GETNext, FSQ): +7.58% Acc@5 (JO); (RNN, FSQ): +51.36% Acc@5 (JO).
  • Travel time estimation (MulT-TTE, Porto): MAE reduced by 21.18% (PO/JO).
  • Trajectory recovery (TrajBERT, Porto): MAE reduced by 34.96% (PO/JO).
  • Map-matching (GraphMM, Tencent): +69.91% Accm (JO) — largest single gain.
  • Trajectory user linking (S2TUL, FSQ): +35.57% Acc@5; (DPLink, FSQ): +25.66% Hit@5.
  • Mobility intention prediction (LIMP, Beijing): +2.38% AUC — smallest gain, near-saturated baseline.
  • Trajectory generation (DSTPP, Earthquake): +22.27% MAE (DA).
  • Trajectory representation (CACSR, FSQ): +12% Acc@5 (DA+JO).
  • Ablation with score-stripped memory: \(\Delta\)Acc@5 < 0.5%, confirming necessity of reward-signal-guided reflection.

Limitations

  • Scope bounded by UniEnv: tasks outside the 18-model/5-task registry are refused rather than generalized to; extensibility requires manual integration of new models.
  • Sequential joint optimization: DA must stabilize before PO begins; direct combination is reported to fail, but the reason is not fully analyzed.
  • No cross-region transfer: models must be trained from scratch per dataset/region due to geospatial heterogeneity; the framework automates this but does not solve the underlying transfer learning problem.
  • LLM cost and latency: each optimization iteration calls an LLM API; the paper does not report wall-clock time or cost comparisons against manual tuning.
  • Self-instruct query set: the 300 evaluation queries are synthetically generated from 5 seed queries, which may not capture real user query diversity.

Relevance to Harnesses / Meta-Harnesses

TrajAgent is a domain-specific meta-harness: it acts as an orchestration layer that selects, configures, executes, and iteratively optimizes task-specific sub-models without human intervention, closely mirroring the structure of general ML meta-harnesses like AutoML pipelines and HuggingGPT. The Collaborative Learning Schema is particularly notable as a harness design pattern—it demonstrates how a high-level reasoning agent can close the loop over low-level training runs, replacing hand-tuned hyperparameter sweeps with LLM-guided search. UniEnv's unified interface (standardized data, model, and task APIs) is a direct instance of the "unified execution environment" pattern that recurs across meta-harness designs, and the four-stage agentic workflow (understand → plan → execute → summarize) maps onto the canonical harness pipeline structure, making TrajAgent a concrete, evaluated case study of these principles applied to a scientific computing domain.