Human-AI Collaboration in Mathematics Education: Investigating the Impact of Large Language Models on Students Mathematical Reasoning and Critical Thinking

Abstrak

The integration of Large Language Models (LLMs) into mathematics education has sparked debate regarding their effect on students’ cognitive development, particularly in mathematical reasoning and critical thinking. This study investigates the comparative impact of three instructional conditions: traditional instruction (TI), LLM-assisted learning (LLM-AL), and human-AI collaborative learning (HAICL) on 240 secondary school students across six schools in Indonesia. Employing a quasi-experimental pretest-posttest control group design, this research measures mathematical reasoning (MR) and critical thinking (CT) using validated instruments with inter-rater reliability of 0.89. Results demonstrate that HAICL significantly outperforms both TI and LLM-AL, with pretest-posttest gain scores of 32.4% for MR and 28.7% for CT, compared to LLM-AL (18.2% MR, 15.3% CT) and TI (12.1% MR, 9.8% CT). Repeated measures ANOVA reveals significant time × group interaction effects (F(2,237) = 24.36, p < 0.001, η² = 0.17). However, K-Fold Cross-Validation (k=5) of predictive models for reasoning improvement shows that LLM-only configurations exhibit overfitting (training accuracy 94.2%, validation 78.3%), while HAICL-based models demonstrate robust generalization (training 86.4%, validation 84.1%). These findings indicate that human-AI collaboration mitigates over-reliance on generative solutions and enhances metacognitive monitoring. This research contributes to understanding optimal integration of LLMs in mathematics pedagogy and emphasizes the necessity of collaborative frameworks over autonomous AI tutoring.

Identifier

• OpenAlex: https://openalex.org/W7166870206
• DOI: https://doi.org/10.61797/ijaaiml.v6i1.768