Teaching LLMs to Plan: Logical CoT Instruction Tuning for Symbolic Planning

<p>Large Language Models (LLMs) like GPT and LLaMA have shown remarkable general capabilities, yet they consistently hit a critical wall when faced with <strong>structured symbolic planning</strong>. This struggle is especially apparent when dealing with formal planning representations such as the <strong>Planning Domain Definition Language (PDDL)</strong>, a fundamental requirement for reliable real-world sequential decision-making systems.</p><p>In this episode, we explore <a href="https://arxiv.org/abs/2509.13351" target="_blank" rel="noopener noreferer"><strong>PDDL-INSTRUCT</strong></a>, a novel instruction tuning framework designed to significantly enhance LLMs&#39; symbolic planning capabilities. This approach explicitly bridges the gap between general LLM reasoning and the logical precision needed for automated planning by using <strong>logical Chain-of-Thought (CoT) reasoning</strong>.</p><p><strong>Key topics covered include:</strong></p><ul><li><strong>The PDDL-INSTRUCT Methodology:</strong> Learn how the framework systematically builds verification skills by decomposing the planning process into explicit reasoning chains about <strong>precondition satisfaction, effect application, and invariant preservation</strong>. This structure enables LLMs to self-correct their planning processes through structured reflection.</li><li><strong>The Power of External Verification:</strong> We discuss the innovative two-phase training process, where an initially tuned LLM undergoes CoT Instruction Tuning, generating step-by-step reasoning chains that are validated by an external module, VAL. This provides ground-truth feedback, a critical component since LLMs currently lack sufficient self-correction capabilities in reasoning.</li><li><strong>Detailed Feedback vs. Binary Feedback (The Crucial Difference):</strong> Empirical evidence shows that <strong>detailed feedback</strong>, which provides specific reasoning about failed preconditions or incorrect effects, consistently leads to more robust planning capabilities than simple binary (valid/invalid) feedback. The advantage of detailed feedback is particularly pronounced in complex domains like Mystery Blocksworld.</li><li><strong>Groundbreaking Results:</strong> PDDL-INSTRUCT significantly outperforms baseline models, achieving planning accuracy of up to <strong>94%</strong> on standard benchmarks. For Llama-3, this represents a <strong>66% absolute improvement</strong> over baseline models.</li><li><strong>Future Directions and Broader Impacts:</strong> We consider how this work contributes to developing more <strong>trustworthy and interpretable AI systems</strong> and the potential for applying this logical reasoning framework to other long-horizon sequential decision-making tasks, such as theorem proving or complex puzzle solving. We also touch upon the next steps, including expanding PDDL coverage and optimizing for optimal planning.</li></ul><p><br></p>