Eliciting Reasoning in Language Models with Cognitive Tools

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper introduces "cognitive tools" — a small set of modular, prompt-driven reasoning operations (understand question, recall related, examine answer, backtracking) exposed to an LLM via a standard tool-calling interface. Without any fine-tuning, this approach substantially improves mathematical reasoning across both open-weight and closed models, with GPT-4.1 + cognitive tools surpassing o1-preview on AIME 2024 (53% vs 44.6%).

Problem

Post-training methods like RL (DeepSeek-R1 style) are the dominant approach for eliciting reasoning in LLMs, but they are resource-intensive and opaque. Cognitive prompting (monolithic structured prompts) improves reasoning but lacks modularity, causing interference between reasoning stages and enforcing a rigid operation order. There is no lightweight, training-free mechanism that surfaces latent reasoning capabilities from base/instruct models while maintaining interpretability.

Method

Four cognitive tools are implemented as isolated LLM self-calls within a standard agentic tool-calling loop: - understand_question: prompts the LLM to decompose the problem and identify relevant theorems/properties. - recall_related: retrieves analogous solved problems from the model's parametric knowledge to guide solution strategy. - examine_answer: reviews the current reasoning trace for flaws, miscalculations, or violated constraints (self-reflection). - backtracking: summarizes the current trace, identifies the incorrect step, and explores alternative reasoning paths (analogous to MCTS).

Each tool is executed by the same LLM instance in a sandboxed context with its own prompt template, and its output is injected back into the main generation loop. The LLM autonomously selects which tools to call and when (no forced ordering). A code-execution capability is also optionally included as an additional modular tool. Evaluation uses pass@1 across AIME 2024 (30 samples), MATH 500, AMC (83 samples), and Smolbenchmark-math (50 samples); results averaged over ≥8 runs with Welch's t-test for significance.

Key Contributions

• A cognitive-architecture-grounded, training-free framework that maps modular cognitive operations onto LLM tool calls.
• Empirical demonstration that cognitive tools consistently outperform both the base model and cognitive prompting across four math benchmarks and six models (Qwen2.5-7/32B, Llama3.1-8B, Llama3.3-70B, GPT-4.1).
• Evidence that GPT-4.1 + cognitive tools (53% AIME 2024) surpasses o1-preview (44.6%) with zero additional training.
• Theoretical contribution to the debate on RL vs. pre-training as sources of reasoning: results suggest latent capabilities exist in base/instruct models and can be surfaced through structured modular prompting.
• Increased interpretability: each reasoning step maps to a named cognitive operation.

Results

• Smolbenchmark (math): cognitive tools beat cognitive prompting across all models; gains over baseline range from +4.2% (Qwen2.5-7B: 75.8→80.0%) to +27.2% (Llama3.3-70B: 52.8→80.0%).
• AIME 2024: Qwen2.5-32B: 17.2→32.1%; Llama3.3-70B: 13.1→29.8%; GPT-4.1: 32→53% (vs. o1-preview 44.6%).
• MATH500: Qwen2.5-32B: 74.1→81.8%; Llama3.3-70B: 57.0→74.7%.
• AMC: Qwen2.5-32B: 52.6→62.7%; Llama3.3-70B: 33.0→51.0%.
• Cognitive tools consistently outperform CoT and code-equipped baselines on aggregate (e.g., Llama3.3-70B avg: baseline 34.4, CoT 43.1, code 45.3, cognitive tools 51.8).
• All individual-tool improvements over baseline on Smolbenchmark are statistically significant (p < 0.05).

Limitations

• Cognitive tool prompts are manually designed and evaluated primarily on mathematical reasoning; generalization to other task types (coding, factual QA, etc.) is untested.
• Prompt templates may not transfer well to model families outside those tested without additional prompt engineering.
• Automated discovery of task-appropriate cognitive operations is left to future work.
• No integration with RL post-training explored; the two approaches are not compared in a combined setting.
• AIME 2024 test set is small (30 samples), which limits statistical power for that benchmark specifically.

Relevance to Agentic AI / LLM Agents

This paper directly extends the agentic tool-calling paradigm inward: instead of routing to external APIs, the tools encapsulate the LLM's own internal reasoning steps, creating an interpretable self-directed reasoning loop. It is directly relevant to agent design questions about how to structure multi-step reasoning without fine-tuning — the cognitive tools framework can be dropped into any existing tool-calling agent stack. The work also challenges the assumption that RL post-training is necessary for strong reasoning, suggesting that well-structured agentic scaffolding can unlock latent capabilities in standard instruct models, which has implications for resource-constrained agent deployments. The backtracking tool in particular operationalizes MCTS-style search within a tool-call loop, connecting tree-search planning literature with practical LLM agent architectures.