Content-Based Smart E-Mail Dispatcher Using Large Language Models¶

🕒 Published (v1): 2026-06-25 04:34 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

A LangChain agent system that reads departmental email inboxes, uses LLM-based content analysis to classify emails by target audience (student batch year or faculty), and automatically dispatches them to the appropriate WhatsApp groups. Gemini achieves 93.5% and ChatGPT 92% exact-match group classification accuracy on a 1,000-email dataset. No labeled training data is required.

Problem¶

Staff in engineering college departments manually read incoming emails and forward them to correct WhatsApp groups—a process that is error-prone, time-consuming, and burdens productivity. Keyword-based routing (existing tools like Pabbly/Zapier) lacks semantic understanding needed to discriminate audience (e.g., faculty-only circulars vs. all-student announcements).

Method¶

Three-stage pipeline: (1) Data capture — read email body and attachments from inbox; (2) Content analysis — a LangChain agent submits email text plus a structured prompt (listing available WhatsApp groups by batch year + faculty) to an LLM API, which returns the set of target groups; (3) Dispatch — custom tool posts the email and attachments to the LLM-specified WhatsApp groups. Classification is zero-shot (no fine-tuning); the prompt encodes group definitions and routing rules as context. Accuracy is defined as exact set match: any extra or missing group counts as misclassification.

Key Contributions¶

Zero-shot LLM-based email routing that handles multi-label group selection without a labeled dataset or retraining when new batches are added.
LangChain agent framework with custom WhatsApp dispatcher tool, enabling step-by-step task decomposition (categorize → identify groups → dispatch).
Comparative evaluation of four LLMs (Gemini, ChatGPT, DeepSeek, Perplexity) on a domain-specific 1,000-email corpus under strict exact-match criteria.
Ablation argument establishing why ML/DL alternatives are impractical: multi-label prediction difficulty, annual group schema changes, and inferior NLP benchmarks vs. LLMs.

Results¶

Gemini: 93.5% exact-match accuracy
ChatGPT: 92.0%
Perplexity: 82.5%
DeepSeek: 80.0%
Primary failure mode: faculty-only circulars (valuation/revaluation) incorrectly recommended to student groups; batch series without year mapped correctly.

Limitations¶

Single-institution deployment; generalizability to other organizational structures untested.
Exact-match metric is extremely strict (off-by-one group = fail), making reported accuracy figures pessimistic but also hard to compare across tasks.
No latency, cost, or throughput figures reported.
WhatsApp API reliability and rate limits not discussed.
Prompt robustness to adversarial or ambiguous email content not evaluated.
Dataset curation details (sources, class distribution) are sparse.

Relevance to Harnesses / Meta-Harnesses¶

This paper implements a minimal single-level agentic harness: a LangChain orchestrator wraps LLM calls with custom tool definitions (the WhatsApp dispatcher) and routes control flow based on LLM output—exactly the pattern meta-harnesses generalize and automate. It is a concrete, narrow-domain instance of the agent-as-router pattern, where the harness itself is hand-coded rather than generated or composed dynamically. For meta-harness researchers, the paper illustrates the practical ceiling of prompt-only orchestration in a well-scoped domain (93.5% exact match) and surfaces the open problem of dynamic schema changes (new batch groups each year) that a self-adaptive meta-harness would need to handle automatically.