Skip to content

Discourse Graph Guided Document Translation with Large Language Models

🕒 Published (v1): 2025-11-10 15:48 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TransGraph is a two-stage, graph-conditioned document-level machine translation framework that first builds a directed discourse graph over document chunks encoding 10 RST-inspired relation types, then translates each chunk conditioned solely on its in-neighborhood in that graph. It consistently outperforms both single-pass and multi-agent baselines (TransAgent, DeLTA) on BLEU, COMET, and terminology accuracy across three benchmarks and six languages, at roughly one-third the token cost of agentic systems.

Problem

Full-document MT with LLMs degrades on long-range coherence because context windows are bounded and sequential context is undifferentiated. Existing agentic MT harnesses (TransAgent, DeLTA) mitigate this with multi-agent orchestration and persistent multi-granularity memory stores, but they incur high token overhead (~104k–171k tokens/document) and are sensitive to memory retrieval placement. Simple sequential-window context lacks structure, causing terminology drift across non-adjacent chunks.

Method

Stage 1 — Document Preparation. The source document is chunked in sentence-boundary-preserving windows of \(T = 100\) tokens. A directed labeled graph \(G = (V, E, \ell)\) is built over chunk pairs within a sliding window \(w\): for each pair \((i, j)\) with \(j - i \le w\), an LLM is prompted to classify the relation as one of 10 types (\(\ell \in \mathcal{R}\)) drawn from RST — including Entity-Coreference, Background→Core, Motivation→Method, Terminology Definition, Cause→Effect, and No-Relation (discarded). This step is model-agnostic and runs entirely via prompting.

Stage 2 — Iterative Chunk Translation. Chunks are translated in index order. For target chunk \(c_j\), its in-neighborhood is retrieved: \(N^-(j) = \{i \mid (c_i, c_j) \in E\}\), capped at 5 neighbors (ties resolved by proximity). A structured context package \(C_j = \{(i, c_i, \ell(c_i, c_j)) : i \in N^-(j)\}\) is assembled, giving the LLM both chunk content and its discourse role. The translation is \(\hat{y}_j = T(c_j \mid C_j)\), and the output document is \(\hat{Y} = \hat{y}_1 \| \cdots \| \hat{y}_N\).

Key Contributions

  • Discourse-graph-conditioned DocMT pipeline that encodes 10 RST-inspired relation types for selective, label-aware context injection.
  • Demonstration that structured discourse context strictly dominates generic sequential context expansion (ablations: \(-\text{REL}\), \(\text{SEQ}\), \(\text{FIXED}\)).
  • Token-efficient alternative to multi-agent MT harnesses: ~59k tokens/document vs. ~104k (TransAgent) and ~171k (DeLTA) at higher translation quality.
  • Backbone-agnostic design validated across Qwen3 (8B/14B/32B), Llama3.1-8B, and Ministral-8B.
  • Quantitative graph analysis: relation labeling accuracy (Qwen3-32B: 94.54% En, 90.45% Zh) and cross-lingual graph consistency (76.81% En↔De, 87.15% En↔Zh).

Results

  • GuoFeng (Qwen3-32B): TransGraph achieves d-BLEU 32.83 and d-COMET 90.47 vs. 26.45/81.23 (Sent-MT), 31.28/89.98 (DeLTA); +6.38 BLEU and +9.24 COMET over sentence baseline.
  • BWB terminology accuracy (Qwen3-32B): 61.72 vs. 53.21 (TransAgent), 52.03 (DeLTA), 51.05 (\(-\)REL variant).
  • ACL60/60 terminology accuracy (Qwen3-32B): 78.65 vs. ~74–75 for both agentic baselines; d-COMET 89.78.
  • Token cost: TransGraph ~59k tokens, 42 LLM calls vs. TransAgent ~104k, DeLTA ~171k; 1-pass DocMT cheapest (~1.8k, 1 call) but worst quality.
  • Exception: Qwen3-8B on GuoFeng — TransGraph tops d-BLEU (26.24) but trails DeLTA on d-COMET (79.29 vs. 81.80), indicating shallow-backbone limits.
  • Graph accuracy (Qwen3-32B): 94.54% En, 89.09% De, 90.45% Zh; degrades to ~78–87% at 8B scale.

Limitations

  • Relation labeling accuracy degrades at small model sizes (Qwen3-8B: 77.72% De, 80.45% Zh), potentially introducing misleading context for smaller backbones.
  • Window parameter \(w\) constrains relation discovery to local pairs; the paper reports only ~35% of total relations lie within 5-nearest neighbors, and the remaining 65% are captured by graph traversal — but a poorly chosen \(w\) could still miss important long-range dependencies.
  • Cohesion evaluation uses LLM-as-Judge, which inherits model biases and cannot be treated as ground truth.
  • Evaluated on three benchmarks with limited domain diversity (technical presentations, Chinese web novels); generalization to other language pairs and domains is unverified.
  • Stage 1 graph construction adds its own LLM call overhead (not zero-cost), though still cheaper than full agentic pipelines.

Relevance to Harnesses / Meta-Harnesses

TransGraph is a structured orchestration harness: it decomposes a complex LLM task (document translation) into a deterministic multi-stage pipeline — chunking agent, relation-labeling agent, graph traversal, translation agent — with explicit data flow between stages. Its direct comparison against multi-agent harnesses (TransAgent, DeLTA) with persistent memory stores makes it a concrete case study in harness design tradeoffs: heavyweight memory-managed orchestration versus lightweight graph-conditioned pipelines. The finding that structured selective context outperforms exhaustive multi-agent retrieval at \(\sim 3\times\) lower token cost is a strong empirical argument for graph-structured harness coordination over stateful agent memory. The ablations (\(-\text{REL}\), \(\text{SEQ}\), \(\text{FIXED}\)) additionally isolate which harness components (coherent chunking, graph topology, relation labels) drive quality, offering a reusable template for designing domain-specific orchestration harnesses.