TY  - JOUR
AU  - Raina, Aryan 
AU  - Gupta, Shiwani 
AU  - Jadhav, Jagruti 
AU  - Bhonde, Sampada 
AU  - Mathur, Shilpa 
AU  - Maha, Anand 
AU  - Sankhe, Pranjali 
PY  - 2026
TI  - Empowering Small Language Models With Chain of Thought and Parameter Efficient Fine Tuning for Efficient Deep Reasoning
JF  - Journal of Computer Science
VL  - 22
IS  - 8
DO  - 10.3844/jcssp.2026.2617.2632
UR  - https://thescipub.com/abstract/jcssp.2026.2617.2632
AB  - Small Language Models (SLMs) run faster and fit on modest hardware, yet solving multi-step logic problems has traditionally been difficult for them. This work investigates a systematic, multi-model framework that combines Chain-of-Thought (CoT) prompting with Low-Rank Adaptation (LoRA) parameter-efficient fine-tuning, and measures the contribution of each component independently. We evaluate three SLMs (TinyLlama-1.1B, Phi-3-mini-4k-instruct, and Qwen2.5-1.5B) using 1,164 training samples drawn from GSM8K, Microsoft Orca-Math-Word-Problems-200k, and OpenAI HumanEval, and evaluate on 500 GSM8K test problems. Phi-3-mini-4k-instruct achieves the strongest overall gains, reaching 16.0% exact match and 61.7% step accuracy under +CoT+LoRA, a 95% improvement over its own baseline. The decline observed under CoT-only prompting and the subsequent recovery under CoT+LoRA were statistically significant (McNemar test, p<0.05). A notable finding is that CoT prompting applied without fine-tuning degrades Phi-3-mini’s exact match score by approximately 44% due to an instruction-format conflict that LoRA fine-tuning resolves. Our ablation shows that neither technique alone produces peak performance; their combination does. The framework trains fewer than 0.05% of model parameters and runs end-to-end on a single consumer-grade NVIDIA RTX 4500 Ada GPU with 24 GB VRAM, making capable SLM-based reasoning deployable without datacenter-scale infrastructure.