Artificial intelligence (AI)-powered chatbots provide problematic medical advice in roughly half of cases, a study published in the medical journal BMJ Open has revealed.
Researchers from the US, Canada, and the UK assessed five popular AI platforms: ChatGPT, Gemini, Meta AI, Grok, and DeepSeek. Each chatbot was presented with ten questions across five health categories.
Nearly 50% of the responses were deemed problematic, with around 20% rated as highly problematic.
The study found that chatbots performed relatively well with closed questions and tasks, particularly on topics like vaccines and cancer.
However, their performance declined with open-ended questions and tasks, as well as on subjects such as stem cells and nutrition.
Responses were often delivered with confidence and certainty, yet none of the chatbots provided fully complete and accurate reference lists for any of their answers, according to the study.
