projects

DYNAMIC REWARD AGENTS FOR LLM REINFORCEMENT LEARNING

This thesis project investigates whether dynamically regenerating an LLM-based reward rubric during reinforcement learning (RL) training can sustain improvements in LLM reasoning beyond what static reward mechanisms achieve. Using Group Relative Policy Optimization (GRPO) and the Phi-4 (14B) model, five training conditions are compared on the MMLU-Pro reasoning benchmark — from an untrained baseline and supervised fine-tuning to RL with a static scalar reward, a static LLM judge, and a fully dynamic LLM judge whose evaluation criteria evolve as the policy improves.

Team members: Raymond Carl

LLM-AS-JUDGE FOR SCIENTIFIC CREATIVITY SCORING

This project investigates prompt engineering strategies for using large language models (LLMs) as automated judges to predict human-assigned creativity scores on the Scientific Creative Thinking Test (SCTT). Through a systematic ablation study, the project compares zero-shot, few-shot, chain-of-thought, and multi-dimensional rubric-based prompting approaches, examining how prompt specificity and reasoning effort affect regression performance against psychometrically derived creativity scores.

Team members: Stanley Nurnberger

AUTOMATED DISCOURSE ANALYSIS OF REASONING PATTERNS IN SCIENCE CLASSROOMS

This project investigates automated classification of reasoning components (RC) and utterance types (UT) in science classroom dialogue using LLM-based data augmentation and fine-tuned transformer models. By probing students' reasoning patterns with a revised 4-class RC scheme (ER, SR-D, SR-I, NA), the project analyzes co-occurrence and sequential discourse patterns, tracks cognitive complexity over lesson time, and identifies instructional moves associated with higher-order student reasoning.

Team members: Mukhesh Raghava Katragadda, Raymond Carl

CREATIVITY ASSESSMENT

This research aims to develop an automated creativity assessment using various AI/ML methodologies including Poly-Encoder, LLM-based regression methods, and pair-wise comparison-based ranking techniques. The project focuses on evaluating the creativity of student-generated ideas in scientific research settings, providing insights into the effectiveness of different assessment approaches and their potential applications in fostering creativity in learning environments.

Team members: Phillip Gregory, Sam Grouchnikov, Stanley Nurnberger

Supported by Interdisciplinary Seed Grant, Kennesaw State University

AI-BASED AUTOMATION FOR CLASSROOM DISCOURSE ANALYSIS (ADAS)

This project aims to develop an Automated Discourse Analysis (ADAS) to enhance science classroom discourse by accurately distinguishing between knowledge construction and reproduction, crucial for fostering scientific reasoning. The project seeks to overcome current limitations in AI's ability to recognize epistemic work in educational settings. Within this project scope, we expand training data, optimize various AI models for discourse analysis, measure the models' confidence in predictions, and pilot the system with diverse secondary science teachers. The goal is to provide teachers with reliable, evidence-based feedback to improve discourse patterns and reduce achievement gaps, ultimately supporting complex pedagogical reasoning in science education.

Team members: Mukhesh Raghava Katragadda, Raymond Carl, Dr. Soon Lee

Supported by Interdisciplinary Seed Grant, Kennesaw State University

EDUCATIONAL QUESTION GENERATION (YOULEQD)

This study introduces the YouTube Learners' Questions on Bloom's Taxonomy Dataset (YouLeQD), which compiles questions posed by learners in YouTube lecture video comments. The research develops two RoBERTa-based classification models that utilize Large Language Models to identify questions and assess their cognitive complexity according to Bloom's Taxonomy. By analyzing the cognitive complexity of learner-posed questions and their interaction metrics, the study provides valuable insights for developing AI models for education.

Team members: Nong Ming, Sachin Sharma

Supported by Interdisciplinary Seed Grant, Kennesaw State University

DOMAIN QUESTION MAPPING (DQM)

This project develops an innovative approach to constructing Domain Question Maps (DQMs) to address challenges in automatically creating concept maps from unstructured educational materials. By leveraging publicly available question-answering datasets, the study fine-tunes pre-trained language models for question generation and uses a textbook's hierarchical outline to train a specificity classification model. This method formulates specific questions aligned with learning objectives, enhancing knowledge representation and learner engagement. DQMs effectively generate educational questions and identify hierarchical relationships, facilitating personalized and adaptive learning.