AI could worsen existing health inequities

Data-driven technologies like artificial intelligence (AI) are powerful tools demonstrating potential in the diagnosis and treatment of diseases such as skin cancer. Yet these could inadvertently worsen the health inequalities experienced by minority ethnic groups if current challenges such as biased algorithms, poor data collection and a lack of diversity in research and development are not urgently addressed.

These are the findings from a new white paper by Imperial College London’s Institute of Global Health Innovation (IGHI), which calls for further research and transparent discussion on the creation and use of these technologies in health care.

The white paper reviewed academic literature and policy evidence to identify the issues and opportunities for AI and data-driven technologies to improve the health and care of minority ethnic groups, who generally experience poorer health than the wider population, as emphasised by the COVID-19 pandemic. The study included interviews with a range of experts in the UK and internationally across academia, industry, NHS policy and practice, legal and regulatory bodies, patient-facing organisations and charities.

Artificial intelligence systems are created by combining large amounts of data, for example from research studies or the Internet. The information is then used to ‘train’ a computer programme or algorithm to make decisions based on the data. For example, using data, AI algorithms can create ‘risk scores’ to predict which patients might be likely to develop certain diseases in the future.

Yet if much of this data is unrepresentative of minority ethnic groups and focuses predominantly on, for example White participants, then these systems are more likely to make decisions which exclude diverse communities.

The report presents evidence of this racial bias in AI, demonstrating how minority ethnic groups can be underserved by technology. For example, facial recognition systems have shown to be up to 19% less accurate at recognising images of Black men and women compared to White individuals. Such bias is also seen in AI when used in the detection and treatment of health conditions such as skin cancer. Images of White patients are predominantly used to train algorithms to spot melanoma which could lead to worse outcomes for Black people through missed diagnoses.

The authors argue unconscious and conscious bias in AI is partly fuelled by the lack of diversity in academia, among AI developers and at strategic levels of the health system and government.

Experts interviewed for the report also voiced concern that the lack of diversity in the AI workforce could lead to solutions which are not fully representative of all users’ needs. The report advises that improving representation in this industry must go further than addressing recruitment processes. Rather, the issue should also be tackled at all stages of education from primary school to postgraduate level.

The authors highlight examples of good practice seen across health and social care in both the UK and USA and propose actions to be undertaken across academia, among AI developers and at strategic levels of the health system and government to work towards developing a system that facilitates the development of more representative AI technologies, designed with equity in mind. At the national level, more is being done to tackle some of these inequities in healthcare. The Government recently announced two independent reviews, following the publication of the “Levelling Up” White Paper, to tackle health disparities, including a review into the health impact of potential bias in medical devices.

Dr Saira Ghafur, digital health lead at the Institute of Global Health Innovation, said: “AI has tremendous potential for healthcare system delivery. However, our white paper shows how it can exacerbate existing health inequities in minority ethnic groups. By working across government, healthcare and the technology sector, it is crucial we ensure that no one is left behind.”

Lord James O’Shaughnessy, Visiting Professor at the Institute of Global Health Innovation, said: “Tackling health inequality is one of the major challenges of our time. Advances in AI and machine learning give us new tools to tackle this challenge, but our enthusiasm must be tempered by a realistic appraisal of the risks of these technologies inadvertently perpetuating inequalities.

“This paper explains how these risks could manifest and makes concrete proposals about how to mitigate them. Policymakers should heed the lessons of the report so that the wonderful advances in computer science can benefit those who most need it.”

Based on this research, the scientists make a series of recommendations to better enable AI for minority ethnic communities. These include:
– Involving patients and the public in all areas of AI technology development.
– Creating governance systems, legislation and regulation in AI which protect data and citizen’s rights.
– Developing a regulatory framework to ensure algorithms are tested on and appropriate for minority ethnic groups to reduce bias in datasets.

Professor Ara Darzi, co-director of the Institute of Global Health Innovation, said: “Our new white paper identifies some important future considerations to maximise the potential of AI for the health of our entire nation. We hope that our report can be a catalyst for the important conversations that need to be had as we increasingly use AI in healthcare.”