AI could perpetuate health inequalities

The Physiological Society launches new report warning of the potential that AI has to perpetuate health inequalities: The Physiological Society has launched its latest report, ‘From ‘Black Box’ to Trusted Healthcare Tools’ in the Houses of Parliament.

This report places a spotlight on the role of Artificial Intelligence (AI) in healthcare and the necessity of integrating physiological evidence into its application to prevent health inequalities.

The Society’s primary focus is exploring ways to establish the UK as a leading ‘science superpower’ in AI and health. A crucial aspect of this ambition is the safe and responsible deployment of AI tools in healthcare. However, The Society raises concerns about potential challenges and risks, ranging from inaccurate diagnoses to the exacerbation of health inequalities due to biased data and access, which could result in patient deaths.

The Society emphasises that AI healthcare tools are often developed, approved, and implemented without sufficient physiological input. Based on consultations with over 30 experts in the field, The Society proposes that limited incorporation of physiological evidence in the development of AI tools undermines trust, poses applicability challenges, and, at worst, lead to the identification of spurious correlations that could potentially harm patients.

As the Government announces that the UK will host the first major global summit on AI safety, The Society suggests a comprehensive approach rooted in establishing a ‘Physiology & AI Framework’, prioritising ‘physiological plausibility’ in research funding mechanisms, and including physiological evidence in the regulatory approval process. ‘Physiological plausibility’ is a critical concept in the field of AI and healthcare, representing the alignment of AI-driven models, predictions or conclusions with known biological and physiological principles.

To accomplish these aims, The Society proposes a three-step strategic roadmap
• To establish a ‘Physiology & AI Framework’ to set improved guardrails for AI in health.
• To prioritise physiologically plausible AI tools in research funding mechanisms.
• To embed physiological evidence in the regulatory approval of AI tools.

Nevertheless, to capitalise on AI’s potential in healthcare, several challenges, such as inaccurate diagnoses and health inequalities, must be addressed. To tackle these, The Society emphasises the integration of physiological measurements and expertise into AI tools, coupled with the promotion of a research ecosystem that encourages collaboration between biomedical understanding, machine learning systems, and clinical expertise.

The launch event will feature a panel of AI, healthcare, and policy experts. Hosted by Stephen Benn, The Viscount Stansgate; the event heard from Stephen Metcalfe MP (Chair of the All Party Parliamentary Group on AI); Dr Richard Siow from King’s College London & The Physiological Society; Svitlana Surodina, also from King’s College London; Michael Ball from the Medical Research Council; and Katrina Payne from the Turing Institute.

The Physiological Society is committed to collaborating with all stakeholders involved in designing, regulating and implementing AI tools in healthcare.

By the end of 2023, The Society commits to
• support the development and adoption of principles and success criteria that describe physiologically plausible technological applications to clarify AI’s ‘black box’,
• initiate a forum to facilitate regular discussions between physiologists and other key stakeholders to achieve a shared understanding of physiological plausibility and the opportunities and risks associated with AI tools in healthcare,
• begin to outline and deliver a training programme for physiologists, developers and data scientists to establish a shared language and understanding for building physiologically plausible technology by design.

The Physiological Society is dedicated to unlocking the potential of AI in transforming healthcare and improving patient outcomes, increasing healthy lifespan, and reducing the burden on the NHS. By fostering a cohesive and collaborative environment, The Society aims to bridge the gap between AI and physiological evidence in healthcare, pushing the boundaries of what is possible.

Dr Richard Siow, Director, Ageing Research at King’s College London, said,

“As AI gains momentum and captivates the media and the general public, physiologists and AI specialists have a crucial responsibility to collaborate, combining their complementary skillsets to ensure the development of safe, robust, and equitable AI tools, that do not perpetuate health inequalities.

“I can see the transformative power of AI in healthcare. However, its potential can only be fully realised when supported by the scientific rigour of physiology, enabling a deep understanding of how the human body functions in health and disease. The Physiological Society’s creation of an AI & Physiology Framework is an important step to informing this.”

1. Link to report https://www.physoc.org/policy/public-health-and-ageing/aiphysiology/