Skip to content

CAPruner: Conceptual-Adjacent Scene Graph Pruner for Enhancing 3D Spatial Reasoning of Large Language Models

🕒 Published (v1): 2026-04-21 02:43 UTC · Source: Arxiv · Venue: ACL 2026 · link

Why this paper was selected

Scene graph pruning removes irrelevant anchors for LLM 3D spatial reasoning

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CAPruner is a lightweight scene-graph pruner for 3D vision-language tasks that replaces proximity-only KNN pruning with a combined fuzzy-semantic and spatial-distance edge-weighting scheme. It uses node-level supervision (avoiding costly edge-level annotation) to train an MLP that scores pairwise object relations by task relevance. The result is a task-specific pruned scene graph that feeds LLMs with fewer but more pertinent tokens, boosting spatial reasoning accuracy.

Problem

When LLMs process 3D scenes via scene graphs, \(n\) objects yield \(O(n^2)\) pairwise relations, quickly exceeding LLM context limits (e.g., >554 objects/scene in InteriorGS → millions of tokens). Existing pruning (3DGraphLLM's proximity-based KNN) keeps only nearest-neighbor edges, discarding task-critical long-range relations and producing disconnected subgraphs (>67% of ScanNet scenes become disconnected after KNN pruning), both of which degrade spatial reasoning.

Method

CAPruner estimates edge importance as a learned function of two factors:

  1. Fuzzy semantic relevance: For each object \(i\) with category \(c_i\), compute \(s_i = \max_{t \in T} \text{Similarity}(c_i, t)\) over query tokens \(T\). Fuzzy (category-level) matching is used instead of strict matching to tolerate misalignment in 3D point cloud–text alignment.
  2. Spatial proximity: Euclidean distance \(\|P_i - P_j\|_2\) between object centers, weighted per the Maxim of Relation.

Edge weight: \(w_{ij} = f(s_i, s_j, \|P_i - P_j\|_2)\) where \(f\) is a 3-layer MLP (1219 parameters).

Training without edge annotations: Node weights are computed via p-norm aggregation \(v_i = \text{sigmoid}\!\left(\sum_j w_{ij}^p\right)^{1/p}\) and supervised against target-object masks using a weighted MSE loss: $\(\mathcal{L} = \frac{1}{|O|}\sum_{i\in O}(v_i-1)^2 + \frac{1}{|\tilde{O}|}\sum_{i\in \tilde{O}} v_i^2\)$ encouraging high weights on edges incident to target objects and low weights elsewhere. At inference, KNN selection is applied on learned edge weights per node (2 edges kept per node, matching 3DGraphLLM's budget).

Key Contributions

  • Empirical finding that removing task-critical relations from scene graphs consistently degrades LLM spatial reasoning, and that KNN pruning breaks graph connectivity in >67% of ScanNet scenes.
  • CAPruner: a task-aware scene-graph pruner combining fuzzy semantic matching with proximity, trained via node-level supervision to avoid quadratic edge annotation cost.
  • Demonstration that task-specific pruning need not preserve global graph connectivity—focusing on the region of interest outperforms MST-based connectivity-preserving alternatives.
  • Inference overhead of 0.75 ms/sample (vs. ~1731 ms for the 8B LLM), making the pruner negligible in cost.

Results

  • ScanRefer (3D visual grounding):
  • CAPruner + Llama-3.2-1B: Acc@0.25 = 55.0 vs. 3DGraphLLM-1B = 52.5 (+2.5); Acc@0.5 = 49.6 vs. 47.5 (+2.1)
  • CAPruner + Llama-3-8B: Acc@0.25 = 61.7 vs. 3DGraphLLM-8B = 60.2 (+1.5); Acc@0.5 = 56.0 vs. 54.6 (+1.4)
  • ScanQA (3D VQA): BLEU-4 improves from 12.2→13.0 (1B) and 12.5→13.2 (8B).
  • SQA3D (situated VQA): EM@1 improves from 52.6→52.8 (1B) and 55.2→56.3 (8B).
  • Token efficiency: CAPruner achieves higher accuracy at 582 avg tokens vs. KNN baseline at 882 avg tokens (34% reduction).
  • Ablation: fuzzy matching (24.57%) > strict matching (24.38%) > BERT similarity (8.57%) > no semantic signal (7.73%) on 3D-VG accuracy; combining fuzzy matching with proximity yields best result.

Limitations

  • Evaluation restricted to indoor scene datasets (ScanRefer, ScanQA, SQA3D); generalization to outdoor or open-vocabulary 3D scenes is untested.
  • Semantic relevance computed only at category level (NYUv2 taxonomy), ignoring fine-grained object properties (color, shape) due to 3D point cloud–text alignment limitations.
  • Fixes the number of retained edges per node to 2 (same as 3DGraphLLM baseline); adaptive budget allocation is not explored.
  • Training still requires target-object annotations; scenes with ambiguous or multiple targets may confuse node-weight supervision.
  • Perspective-dependent spatial relations (left/right/front/back) are handled only via proximity as a proxy, which may not capture viewpoint-dependent task requirements correctly.

Relevance to Vision-Language Models

CAPruner addresses a core bottleneck for LLM-based 3D-VL systems: the quadratic scaling of scene-graph tokens, which limits applicability to real-world scenes. By making pruning task-aware via query-conditioned semantic scoring, it advances the paradigm of structured scene representations for VLMs beyond naive proximity heuristics. The node-level supervision trick is broadly applicable wherever relation-level annotations are impractical, a common constraint in structured VLM grounding tasks. This work is directly relevant to researchers studying grounded scene understanding, 3D spatial reasoning, and efficient long-context multimodal reasoning pipelines.