Skip to content

NeLLCom-Lex: A Neural-agent Framework to Study the Interplay between Lexical Systems and Language Use

🕒 Published (v1): 2025-09-26 15:25 UTC · Source: Arxiv · Venue: EMNLP 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

NeLLCom-Lex is a neural-agent simulation framework that grounds language agents in an existing human lexical system (English color naming) and subjects them to varying communicative pressures through a two-phase Supervised Learning → Reinforcement Learning (SL→RL) pipeline. Agents trained with contextual access during both phases reproduce human-like pragmatic naming behavior and develop more human-like lexicons, supporting causal investigation of lexical semantic change.

Problem

Observational methods (corpus analysis, distributional semantics) cannot establish causal mechanisms of lexical semantic change, and human behavioral experiments cannot span the diachronic timescales required. Existing neural emergent-communication studies start from blank-slate agents, making it impossible to model how a pre-existing real-language lexical system adapts to shifting communicative needs.

Method

The framework extends NeLLCom (Lian et al., 2023) to the lexical domain:

  1. Task: A dyadic referential discrimination game using color chips in CIELAB space. The speaker receives a target color \(c_t\) and two distractors \(c_{d1}, c_{d2}\); it produces a single-symbol color name \(w \in V\) from a fixed vocabulary. The listener receives \(w\) and a shuffled triplet and must identify the target.

  2. Architectures: Both speaker and listener are feed-forward neural networks (FNNs). The speaker encodes each color independently, concatenates embeddings, and predicts \(w\) via a linear classifier. Context-awareness is toggled by zeroing distractor embeddings. The listener scores \(w\) against each color candidate via dot-product similarity followed by softmax.

  3. SL phase: Agents are trained on the English Colors dataset (Monroe et al., 2017; 12.4K human color-naming instances) via cross-entropy loss: $\(\mathcal{L}_S^{\text{sup}} = -\sum_{(c,w)\in D} \log p(w \mid c;\theta_S)\)$

  4. RL phase: Joint speaker–listener optimization via REINFORCE, maximizing communicative success: $\(\mathcal{L}_{S,L}^{\text{comm}} = -\sum \log p(\hat{w} \mid c;\theta_S) \cdot r_L\)$ where \(r_L = \log p(i \mid \hat{w}, \tilde{c};\theta_L)\).

  5. Evaluation metrics: Pragmatic adaptation \(\beta(E_{\text{ctx}})\) (slope from a linear mixed-effects model of word informativeness \(I_w\) on context ease \(E_{\text{ctx}}\)); system informativeness \(I_L = \frac{1}{N}\sum I_w^i\); lexical diversity \(|W|\); semantic drift \(D_L\) (average prototype distance from human lexicon).

Key Contributions

  • NeLLCom-Lex framework: first to ground emergent-communication agents in a real human lexical system using continuous perceptual (CIELAB) inputs rather than symbolic data
  • Systematic ablation of five SL±/RL± context-access pipelines, establishing context as a necessary and sufficient condition for human-like pragmatic behavior
  • Unified measurement of lexical adaptation at both the use level (pragmatic adaptation) and system level (\(I_L\), \(|W|\), \(D_L\)) using metrics from Gualdoni & Boleda (2024)
  • Demonstration that interaction (RL) is required to approach human-like lexical diversity, independent of SL context exposure

Results

  • Speaking accuracy: 79% (context-aware SL) vs. 75% (context-unaware), against human-labeled \(\text{TEST}_{\text{hum}}\)
  • SL+RL+ pipeline: \(I_L = 2.60\), \(|W| = 34.5\) vs. human \(I_L = 2.78\), \(|W| = 49.0\) — closest match
  • SL alone with context (SL+): \(I_L = 3.23\), \(|W| = 12.7\) — pragmatic but over-informative and lexically impoverished
  • No-context pipelines: SL−RL− yields \(I_L = 5.28\), \(|W| = 21.1\) — overly informative, low diversity
  • Pragmatic adaptation \(\beta(E_{\text{ctx}})\) is significantly negative (\(p < 0.001\)) only in context-aware conditions; no significant difference among the three context-aware pipelines
  • Communication accuracy recovers to context-trained levels after just one RL epoch even for context-naive SL agents

Limitations

  • Restricted to a single domain (color naming) and a single within-generation timescale; generalization to other lexical domains or cross-generational transmission is untested
  • Fixed discrete vocabulary \(V\); no open vocabulary or compositional language
  • Only dyadic speaker–listener interactions; no population-level dynamics
  • Grounded in English only; cross-lingual or typologically diverse lexical systems are not explored
  • Text is truncated before §5.2 results (communicative environment variation), so full findings are unavailable here

Relevance to Harnesses / Meta-Harnesses

NeLLCom-Lex is architecturally a two-stage pipeline harness: a supervised grounding stage feeds learned representations into a downstream RL communication stage, with outputs from stage one serving as the initialization state for stage two — a canonical harness composition pattern. The framework exposes clean seams for swapping context conditions, training data, evaluation metrics, and agent architectures, mirroring the modularity goals of software meta-harnesses that chain configurable specialist components. Its ablation methodology — systematically enabling/disabling context at each pipeline stage — is exactly the kind of controlled, multi-condition sweep that harness designs enable and that monolithic models obscure. For researchers building multi-agent orchestration harnesses, NeLLCom-Lex offers a concrete template for how staged SL→RL pipelines can be instrumented to measure emergent properties at both local (per-agent) and global (system-level) granularities.