Google’s Med-PaLM 2, an AI tool designed to answer questions about medical information, has been in testing at the Mayo Clinic research hospital, among others, since April, The Wall Street Journal reported this morning. Med-PaLM 2 is a variant of PaLM 2, which was announced at Google I/O in May this year. PaLM 2 is the language model that underpins Google’s Bard.
The newspaper also mentions Google search made public in May (pdf) showing that Med-PaLM 2 still suffers from some of the accuracy issues that we are already used to seeing in large language models. In the study, doctors found more inaccuracies and irrelevant information in the answers provided by Google’s Med-PaLM and Med-PalM 2 than those of other doctors.
Yet on almost every other measure, such as showing evidence of reasoning, consensus responses, or showing no signs of incorrect understanding, Med-PaLM 2 performed more or less as well as real doctors did.
WSJ reports that customers testing Med-PaLM 2 will control their data, which will be encrypted, and Google will not have access to it.
According to Google’s senior research director, Greg Corrado, WSJ said, Med-PaLM 2 is still in its infancy. Corrado said while he wouldn’t want it to be part of his own family’s “health journey,” he believes Med-PaLM 2 “is taking places in healthcare where AI can be beneficial and multiply them by 10”.
We contacted Google and Mayo Clinic for more information.
