LexChronos: An Agentic Framework for Structured Event Timeline Extraction in Indian Jurisprudence

🕒 Published (v1): 2026-03-02 09:31 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LexChronos is a dual-agent iterative framework that extracts structured event timelines from Indian Supreme Court judgments, using a LoRA-tuned extraction agent and a pre-trained feedback agent in a confidence-driven refinement loop. To overcome the absence of annotated Indian legal datasets, the authors reverse-engineer a 2,000-sample synthetic corpus via DeepSeek-R1 and GPT-4. The pipeline achieves a BERTScore F1 of 0.8751, and GPT-4 prefers structured-timeline-based summaries over raw-text baselines in 75% of cases.

Problem

LLMs applied to Indian Supreme Court judgments treat documents as monolithic unstructured text, failing to capture temporal, causal, and hierarchical event structures essential for downstream tasks (summarization, precedent mapping, judgment prediction). No publicly available event-level annotated dataset exists for Indian jurisprudence, and document-level extraction requiring cross-paragraph co-reference resolution and temporal-causal linkage remains unsolved.

Method

Synthetic dataset construction (reverse-engineering pipeline): 1. Randomly sample a case category from 25 Supreme Court classifications. 2. \(E = \text{PromptE}(\text{case\_category})\) via DeepSeek-R1 or GPT-4 generates a JSON event timeline under the LexChronos Event Schema (LES): {Timestamp, Event, Judge, Precedent}. 3. \(J = \text{PromptJ}(E)\) generates a matching synthetic judgment text, emulating real judicial rhetoric. 4. Repeat 2,000 times to obtain \(\mathcal{D} = \{(E_i, J_i)\}_{i=1}^{2000}\).

Dual-agent iterative extraction: - Extraction agent: LoRA-instruct-tuned open-source LLM (\(\leq 4\)B params) proposes candidate event set \(E_0\) from judgment \(J\). - Feedback agent: pre-trained LLM (\(< 4\)B params) scores \(E_i\) against \(J\) across 7 dimensions (Narrative Relevance, Temporal Accuracy, Chronological Flow, Event Detail, Repetition, Character Identification, Confidence Score), producing critique \(F_i\). - Iteration: \(E_{i+1} = \text{ExtractionAgent}(J, E_i, F_i)\); \(F_{i+1} = \text{FeedbackAgent}(J, E_{i+1})\). - Stopping criteria: (a) patience limit — confidence score fails to exceed best \(S_{\text{best}}\) for 3 consecutive iterations; (b) tolerance threshold — \(S_{i-2} = S_{i-1} = S_i\).

Downstream evaluation: 200 sampled judgments summarized using structured event timelines vs. raw judgment text; GPT-4 acts as pairwise judge across an 8-criterion legal quality checklist.

Key Contributions

• Dual-agent iterative refinement framework with confidence-driven stopping criteria for structured legal event timeline extraction.
• LexChronos Event Schema (LES) with four attributes (Timestamp, Event, Judge, Precedent) validated against 8 canonical Indian Supreme Court judgment components.
• Synthetic annotated corpus of 2,000 Indian Supreme Court judgments (1,000 DeepSeek-R1, 1,000 GPT-4) with gold-standard event-level annotations spanning 25 case categories.

Results

• Best extraction agent achieves BERTScore F1 = 0.8751 on held-out 400-sample test set (8 models evaluated, all LoRA-tuned, \(\leq 4\)B params).
• GPT-4 preferred structured-timeline-based summaries over unstructured baselines in 75% of 200 pairwise comparisons.
• Empirically, 3 iterations provided optimal performance/cost trade-off; patience and tolerance stopping criteria converge reliably within this budget.
• DeepSeek-R1 generated longer judgments (avg 1,010 tokens, 27 events/case, 6 precedents/case) vs. GPT-4 (avg 552 tokens, 19 events/case, 3 precedents/case); DeepSeek-R1 corpus has 2× unique vocabulary size.

Limitations

• Circular evaluation: ground-truth annotations are themselves generated by the same LLMs used for dataset creation; no human-annotated gold standard exists.
• No real judgment evaluation: all extraction and summarization experiments use synthetic documents only; generalization to authentic Supreme Court texts is unvalidated.
• GPT-4 as sole judge: downstream summarization evaluation relies entirely on GPT-4 without human correlation study on this domain.
• Small model constraint: all extraction/feedback agents capped at \(< 4\)B parameters; larger models may qualitatively change the feedback dynamics.
• Single downstream task: only summarization is evaluated; precedent mapping, argument generation, and judgment prediction are proposed but untested.

Relevance to Harnesses / Meta-Harnesses

LexChronos is a direct instance of a multi-agent harness: it wraps two specialized agents (extractor and evaluator) in an explicit orchestration loop with programmatic stopping criteria — the canonical harness pattern of fan-out, score, route-or-halt. The feedback agent functions as a meta-evaluator whose confidence signal governs the orchestrator's control flow, making this a self-refining loop akin to harness-level retry/quality-gate logic. The reverse-engineering dataset pipeline is itself a three-stage harness (select → generate annotations → generate text) composing two LLM calls with structured intermediate representations. For researchers building domain-specific extraction harnesses, LexChronos exemplifies how to design stopping criteria, multi-dimensional scoring rubrics, and LLM-as-judge evaluation protocols within a lightweight (\(< 4\)B param) agentic loop.