Assessing the Accuracy, Readability and Understandability of Websites, ChatGPT, Copilot, and Bard Answers on the Radiation during Pregnancy

INTRODUCTION: The study aims to evaluate the accuracy of answers to frequently asked questions (FAQ) about the impact of radiation during pregnancy on websites, ChatGPT, Copilot, and Bard. Secondly, to assess the readability and understandability of answers.
METHODS: The answers to these questions were scored in terms of accuracy (completely correct, partially correct, incorrect). The Automated Readability Index (ARI), Flesch Reading Ease (FRE), and Gunning Fog Readability (GFR) scores were calculated. The understandability score was assessed using Patient Education Materials Assessment Tool (PEMAT).
RESULTS: The accuracy was calculated as 100% for the websites, 66.67% for ChatGPT, 73.33% for Copilot, and 93.33% for Bard. Readability scores ranking was ChatGPT (ARI=16.15, FRE=24.47, GFR=20.52), Copilot (ARI=14.00, FRE=37.60, GFR=18.27), websites (ARI=13.59, FRE=43.67, GFR=15.56), Bard (ARI=10.92, FRE=48.73, GFR=14.86). ChatGPT's readability was statistically most challenging. PEMAT understandability scores were 79.53% for Bard, below the acceptable limit of 70% for others.
DISCUSSION AND CONCLUSION: While the responses from chat tools and websites may be largely accurate, it is observed that they are not suitable for patients in terms of readability and understandability. Internet information sources should be developed, especially to ensure that the content is understandable by a broad readership.