Reinforcement Learning-Supervised LLM Question Generation from Educational Textbooks

Abstrak

Large Language Models (LLMs) show promise for generating educational questions from textbook content. However, their outputs still need quality control before they can be used in classrooms. This study investigates how prompt constraint design impacts the quality of LLM questions and tests, and whether post-hoc filtering can enhance this process. A total of 566 questions were generated from Indonesian elementary school textbooks using GPT-3.5-turbo and Gemini 2.0-flash, with three different prompt constraint levels (strict, medium, and loose). The experimental results indicate that prompt engineering is the most influential factor. Strict prompts achieved 97.9% answer findability while loose prompts only reached 72.8%, which is a 25% difference. In addition, a Reinforcement Learning (RL)-based supervisor was developed as a proof-of-concept, which achieved 100% findability on accepted questions. The RL-based supervisor demonstrated similar performance compared to a simple rule-based verification method (verifying if the answer appears in the book). The findings suggest that the RL framework could be useful for more complex quality criteria in the future. Moreover, it was also revealed that story problems are approximately 20% harder than factual questions, while GPT-3.5 demonstrated better performance than Gemini 2.0 in terms of findability, achieving 87.5% compared to 84.1%. However, Gemini 2.0 performed better at matching difficulty levels.

Identifier

• OpenAlex: https://openalex.org/W7163738101
• DOI: https://doi.org/10.48084/etasr.17900