27 - A Mixed-Methods Study of AI Use in ABPD Exam Preparation

Abstract:

Purpose: The purpose of this two-fold study was to (1) assess attitudes, beliefs, early adoption patterns, and perceived effectiveness of generative artificial intelligence platforms (GenAIs) used for the American Board of Pediatric Dentistry (ABPD) certification examinations and (2) compare the ability of free versus paid GenAIs to provide educational content using knowledge-based prompts (KBP) and critical thinking prompts (CTP).



Methods: Arm 1 consisted of a cross-sectional REDCap survey distributed via the American Academy of Pediatric Dentistry listserv with postcard consent. Survey items assessed demographics, use and perceived effectiveness of traditional study resources, GenAI adoption, and perceived barriers. Arm 2 evaluated four free/paid GenAIs using standardized KBP and CTP related to pediatric dentistry. GenAI-generated responses were assessed using the validated Artificial Intelligence Study Material Assessment and Reliability Tool (AI-SMART). Data were statistically analyzed.



Results: A total of 414 completed surveys were analyzed (response: 8%; completion=73%). Most respondents relied on academy’s review courses and app-based modules, which were rated as effective. GenAI use for ABPD preparation was reported by under 13percent of respondents. In Arm 2, free and paid GenAIs demonstrated comparable AI-SMART scores (P >.05). However, the quality of evidence was rated as poor for free GenAIs. Across platforms, CTP responses received lower scores than KBP responses (P=.01).



Conclusions: Traditional review methods remain primary for ABPD certification examination preparation. Free and paid GenAI platforms performed similarly for knowledge-based content, but lower critical thinking performance highlights the need for refinement before educational integration in pediatric dentistry.

Identify Supporting Agency and Grant Number: