127 - Physician Management of Dental Trauma Utilizing Large Language Models

Abstract



Purpose: This study was conducted to assess how medical providers in pediatric emergency departments approach traumatic dental injuries (TDIs) when using large language models (LLMs) compared to traditional text guidelines. The aims were to obtain a baseline knowledge of TDI management, and to assess differences in treatment planning and time taken to answer questions between study groups.



Methods: We conducted a three-arm, randomized, survey-based study with medical providers in a Children’s Emergency Department. Participants included medical students, residents, and fellows, attending physicians, and advanced practice providers. Each participant answered 12 case-based questions covering primary and permanent tooth avulsion, permanent tooth fracture, and antibiotic indications. Each participant was assigned to use either no resources, a PDF of the IADT reference guide, or ChatGPT. We recorded how many questions each participant answered correctly, incorrectly, or marked "not sure," as well as how long they took to complete the survey and how confident they felt in their TDI knowledge before starting. 18 providers total participated in this study.



Results: The only outcome to reach significance was a positive correlation between the self-assessment score and % questions correct for the control group using no resources (p=0.003). The median pre-survey self-assessments for confidence in treating TDIs on a scale from 0-10 were: No Resources (6.0), PDF (3.0), ChatGPT (3.3). The No Resource group had a higher ratio of residents and attendings and the ChatGPT group had more medical students and APPs, which may have influenced the confidence rankings.

While not significant, there was increased average accuracy and decreased average time taken when using the ChatGPT vs the PDF. No Resource had the lowest % accurate and fastest completion time.

The areas with the highest % correct answers were primary tooth avulsion and permanent tooth re-implantation timing, storage medium, and splinting. The lowest % correct were antibiotic indications for avulsions and root fractures, the use of Doxycycline, and indications for soft-tissue imaging for tooth fractures.



Conclusion: Although this study was underpowered, it showed that there is variation in how TDIs are treated, even between providers utilizing the same reference materials. Regardless of reference material used, we found gaps in knowledge in multiple areas, particularly antibiotic use. This may indicate a need for a TDI guide geared toward medical providers written by pediatric dentists.

As the technology advances, LLMs may be a useful adjunct for medical providers in the treatment of TDIs, particularly in time-sensitive situations where a consulting dentist is not available. Additional studies are needed to assess how healthcare providers in Emergency Departments can utilize LLMs to retrieve accurate information quickly and improve patient treatment outcomes for dental injuries.