Teaching Large Language Models to Think Twice

Modern large language models struggle with complex mathematical reasoning tasks despite their impressive performance across various natural language processing applications. The primary challenge lies in their inability to detect and correct errors during multi-step problem-solving, where early mistakes cascade through subsequent reasoning steps.

This paper presents a practical framework enabling language models to systematically verify and refine their solutions through structured self-correction. Our approach organizes the problem-solving process into three distinct stages: generating initial solutions with explicit identification of critical reasoning steps, analyzing these solutions for potential errors, and producing corrected final answers.

Comprehensive evaluation demonstrates substantial improvements: 49.9% accuracy on GSM8K (60% relative improvement over 31.2% baseline), 21.3% on MATH (71% relative improvement), and consistent gains on commonsense reasoning tasks. Statistical significance testing confirms these improvements (p < 0.001).

1. Three-Stage Self-Correction Framework (Generator, Critic, Synthesizer) that decomposes problem-solving into distinct computational personas, each optimized for specific cognitive functions.
2. Novel Three-Phase Training Methodology combining supervised fine-tuning on high-quality reasoning traces with reinforcement learning using a Hierarchical Process-Based Reward (HPBR) system.
3. Comprehensive Experimental Validation on GSM8K mathematical reasoning benchmark achieving 60% relative improvement in accuracy (from 31.2% to 49.9%).
4. Detailed Error Analysis revealing 78% correction success rate on computational errors, demonstrating the framework's effectiveness across different error types.
5. Superior Computational Efficiency compared to ensemble methods, using 74% fewer resources while achieving 29% higher accuracy.
6. Metacognitive Capabilities teaching LLMs to develop intrinsic self-correction abilities without requiring external verification systems.