TreeAgent: A Generalizable Multi-Agent Framework for Automated Bias Labeling in Forestry via Compiled Expert Rules and Vision-Language Models

🕒 Published (v1): 2026-06-30 17:16 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TreeAgent is a multi-agent system for automated seven-class tree height bias labeling in forestry remote sensing. It introduces the Decoupled Declarative Decision (D3) framework, which compiles natural-language expert rules into executable decision graphs over a fixed primitive vocabulary, routing samples through deterministic arithmetic predicates or VLM agents at each node. On two held-out NEON sites, TreeAgent achieves 67.6% Macro-F1 at 0.040 min/tree versus 36.2% for a tuned LightGBM baseline and ~5 min/tree for human annotators.

Problem

Expert annotation of per-tree height bias (distinguishing field, LiDAR point-cloud, and CHM error modes) takes 3–5 minutes per tree with significant inter-annotator inconsistency, and does not scale to continental inventories. Prior tabular ML pipelines require labeled data from each ecosystem and cannot encode the conditional perceptual judgments experts make (e.g., crown overlap detection). Naively encoding expert rules as symbolic classifiers tightly couples domain logic to implementation, making threshold revisions expensive re-engineering.

Method

The D3 Framework separates what to decide from how to execute it via two components:

1. Logic Primitive Inventory (LPI): A finite, closed vocabulary \(V = V_\text{det} \cup V_\text{vlm} \cup V_\text{exit}\). Each node \(v = \langle \text{id}(v), \omega(v), \varepsilon(v), \vartheta(v) \rangle\) has an execution type (\(\omega\)), an input signature over tabular fields \(F\) and image modalities \(I\), and a binary evaluation kernel \(\vartheta\).

2. Neural Rule Transpiler (NRT): An LLM \(C_\theta\) that compiles a natural-language expert rule \(\rho\) into a validated JSON tree configuration \(T\) (a DAG over \(V\)) in a single inference call. A deterministic validator checks vocabulary closure (\(O(|N|)\)) before deployment.

At inference, the orchestrator traverses \(T\) via: $\(n_{t+1} = \text{succ}(n_t,\ o_{n_t}(x))\)$ where \(V_\text{det}\) nodes evaluate closed-form arithmetic predicates and \(V_\text{vlm}\) nodes dispatch prompts to a VLM agent. To suppress stochasticity, VLM nodes use \(K=3\) independent samples at temperature \(\tau=0.2\) aggregated by majority vote. The orchestrator is entirely agnostic to the content of \(T\), enabling zero-modification reconfigurability: a rule change is a configuration edit, not a code change.

The instantiation (TreeAgent) encodes a seven-class forestry bias diagnostic. Two VLM node classes handle perceptual tasks: CrownOverlap (bird's-eye CHM rendering) and GroundPcdOutlier (multi-year ground elevation transect).

Key Contributions

• D3 Framework: decoupled declarative architecture with a closed LPI and an LLM-based NRT that compiles expert rules into statically validatable, auditable decision DAGs.
• TreeAgent: instantiation of D3 for forestry bias classification with a majority-vote VLM agent at perceptual nodes.
• Empirical evaluation: cross-ecosystem test (train on OSBS, test on WREF+SRER) showing the agent system substantially outperforms tuned tabular ML, PointNet, and tabular foundation models.
• Prompt ablation for CrownOverlap across four verbosity levels, characterizing the recall–specificity trade-off.
• Rule sensitivity analysis across four semantically plausible phrasings of the NoDiff predicate, selected a priori.

Results

• TreeAgent (frozen headline rule): 67.6% Macro-F1, 0.040 ± 0.007 min/tree.
• LightGBM baseline (tuned, imbalance-corrected): 36.2% Macro-F1.
• Human annotation: ~5.0 ± 2.0 min/tree (reference cost baseline only).
• Per-class recall range: 65.9% (bias 2) to 85.7% (bias 6, No Difference), with unknown class at 72.2%.
• Rule phrasing sensitivity: Macro-F1 varies from 21.9% to 67.6% across four NoDiff predicate variants, all selected a priori—three of four fall below the LightGBM baseline.
• CrownOverlap VLM: V1 prompt achieves 100% recall / 25.0% specificity (adopted as default); V3 achieves highest accuracy (62%) but misses true overlaps; Minimal prompt achieves 100% recall / 7.1% specificity.
• PointNet and all supervised ML ablations (XGBoost, Random Forest, CatBoost, stacking ensemble, TabPFN) are reported but do not match TreeAgent's Macro-F1.

Limitations

• Macro-F1 is sensitive to the exact phrasing of the NoDiff tolerance rule; three of four tested phrasings underperform the LightGBM baseline, indicating the headline result depends on precise rule specification.
• Evaluation uses a single expert annotation per tree; labels are reference annotations, not objective ground truth—reported Macro-F1 measures agreement with one annotator, not labeling correctness.
• CrownOverlap VLM node has low specificity across all prompt variants (max 39.3%), indicating general-purpose VLMs struggle with this perceptual task.
• Dataset is small (283 trees across three sites; 136 train / 147 test); cross-validation is omitted.
• The LPI vocabulary must be pre-specified; rules requiring primitives outside \(V\) cannot be compiled without extending the inventory, breaking the zero-modification guarantee.
• No inter-expert agreement study; calibrating the Macro-F1 ceiling requires multiple annotations per tree.

Relevance to Harnesses / Meta-Harnesses

D3 is a domain-specific meta-harness: it treats the expert decision tree as a structural prior that orchestrates heterogeneous agents (deterministic predicates and VLM calls) through a compiled, validatable DAG, rather than hard-coding control flow in imperative code. The NRT acts as a compilation layer analogous to a rule-to-workflow transpiler, and the LPI defines a closed execution vocabulary that constrains what agents the harness can route to—a pattern directly relevant to harness design for mixed symbolic/neural pipelines. The zero-modification reconfigurability property (rule changes as config edits) addresses a core harness engineering concern: decoupling domain logic from orchestration infrastructure. This paper offers a concrete, evaluated instantiation of the principle that agent scaffolds should separate what decisions to make from how to execute them, with static validation before deployment.