Researchers outline how AI chatbots could be approved as medical devices

Health News
Tags :
Health News
Share This :

Credit: Pixabay/CC0 Public Domain

LLM-based generative chat tools, such as ChatGPT or Google’s MedPaLM have great medical potential, but there are inherent risks associated with their unregulated use in health care. A new Nature Medicine paper by Prof. Stephen Gilbert and colleagues addresses one of the most pressing international issues of our time: How to regulate Large Language Models (LLMs) in general and specifically in health.

“Large Language Models are neural network language models with remarkable conversational skills. They generate human-like responses and engage in interactive conversations. However, they often generate highly convincing statements that are verifiably wrong or provide inappropriate responses,” said Prof. Stephen Gilbert, Professor for Medical Device Regulatory Science at Else Kröner Fresenius Center for Digital Health at TU Dresden.

“Today there is no way to be certain about the quality, evidence level, or consistency of clinical information or supporting evidence for any response. These chatbots are unsafe tools when it comes to medical advice and it is necessary to develop new frameworks that ensure patient safety.”

Challenges in the regulatory approval of large language models

Most people research their symptoms online before seeking medical advice. Search engines play a role in decision-making process. The forthcoming integration of LLM-chatbots into search engines may increase users’ confidence in the answers given by a chatbot that mimics conversation. It has been demonstrated that LLMs can provide profoundly dangerous information when prompted with medical questions. LLM’s underlying approach has no model of medical “ground truth,” which is dangerous.

Chat interfaced LLMs have already provided harmful medical responses and have already been used unethically in “experiments” on patients without consent. Almost every medical LLM use case requires regulatory control in the EU and U.S. In the U.S. the lack of explainability disqualifies them from being “non devices.” LLMs with explainability, low bias, predictability, correctness, and verifiable outputs do not currently exist and they are not exempted from current (or future) governance approaches.

In this paper the authors describe the limited scenarios in which LLMs could find application under current frameworks, they describe how developers can seek to create LLM-based tools that could be approved as medical devices, and they explore the development of new frameworks that preserve patient safety.

“Current LLM-chatbots do not meet key principles for AI in health care, like bias control, explainability, systems of oversight, validation and transparency. To earn their place in medical armamentarium, chatbots must be designed for better accuracy, with safety and clinical efficacy demonstrated and approved by regulators,” concludes Prof. Gilbert.

More information:
Stephen Gilbert et al, Large language model AI chatbots require approval as medical devices, Nature Medicine (2023). DOI: 10.1038/s41591-023-02412-6.

Researchers outline how AI chatbots could be approved as medical devices (2023, June 30)
retrieved 1 July 2023

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *

Popular News Posts


Our Exclusive Products

Our Store

Our mission is simple, offer quality products that make our customers lives better at a reasonable price, while ensuring a hassle-free shopping experience. This means that before we bring a product to market, we test it to ensure that it meets our high-level quality standards. Our customer service team understand our products and can answer most every product related question quickly and efficiently. We strive everyday to meet and exceed our customers’ expectations of quality and support! Should we ever fail to meet this expectation – contact us and we will make it right!

Our Value: We are proud of our product and accomplishments, but we typically don’t shout it from the rooftop. Instead, we prefer to let our product and customers do the talking. Our core values can be recognized in our product. The qualities we strive for include:

Pragmatism: We design simple, useful solutions for common needs
Quality: From design to final product, we strive for durable solutions that work
Originality: Creativity and innovation are what makes our product unique
Design: Our minimalist and bold design language focuses on functionality and simplicity that is timeless

0 +

Top Rated Products

0 +

Happy Customers