New Recommendations to Increase Transparency and Tackle Potential Bias in Medical AI Technologies

Patients will be better able to benefit from innovations in medical artificial intelligence (AI) if a new set of internationally-agreed recommendations are followed.

A new set of recommendations published in The Lancet Digital Health and NEJM AI aims to help improve the way datasets are used to build Artificial intelligence (AI) health technologies and reduce the risk of potential AI bias.

Innovative medical AI technologies may improve diagnosis and treatment for patients, however some studies have shown that medical AI can be biased, meaning that it works well for some people and not for others. This means some individuals and communities may be 'left behind,' or may even be harmed when these technologies are used.

An international initiative called ‘STANDING Together (STANdards for data Diversity, INclusivity and Generalisability)’ has published recommendations as part of a research study involving more than 350 experts from 58 countries. These recommendations aim to ensure that medical AI can be safe and effective for everyone. They cover many factors which can contribute to AI bias, including:

  • Encouraging medical AI to be developed using appropriate healthcare datasets that properly represent everyone in society, including minoritised and underserved groups;
  • Helping anyone who publishes healthcare datasets to identify any biases or limitations in the data;
  • Enabling those developing medical AI technologies to assess whether a dataset is suitable for their purposes;.
  • Defining how AI technologies should be tested to identify if they are biased, and so work less well in certain people.

Dr Xiao Liu, Associate Professor of AI and Digital Health Technologies at the University of Birmingham and Chief Investigator of the study said:

"Data is like a mirror, providing a reflection of reality. And when distorted, data can magnify societal biases. But trying to fix the data to fix the problem is like wiping the mirror to remove a stain on your shirt.

"To create lasting change in health equity, we must focus on fixing the source, not just the reflection."

The STANDING Together recommendations aim to ensure that the datasets used to train and test medical AI systems represent the full diversity of the people that the technology will be used for. This is because AI systems often work less well for people who aren’t properly represented in datasets. People who are in minority groups are particularly likely to be under-represented in datasets, so may be disproportionately affected by AI bias. Guidance is also given on how to identify those who may be harmed when medical AI systems are used, allowing this risk to be reduced.

STANDING Together is led by researchers at University Hospitals Birmingham NHS Foundation Trust, and the University of Birmingham, UK. The research has been conducted with collaborators from over 30 institutions worldwide, including universities, regulators (UK, US, Canada and Australia), patient groups and charities, and small and large health technology companies. The work has been funded by The Health Foundation and the NHS AI Lab, and supported by the National Institute for Health and Care Research (NIHR), the research partner of the NHS, public health and social care.

In addition to the recommendations themselves, a commentary published in Nature Medicine written by the STANDING Together patient representatives highlights the importance of public participation in shaping medical AI research.

Sir Jeremy Farrar, Chief Scientist of the World Health Organisation said: "Ensuring we have diverse, accessible and representative datasets to support the responsible development and testing of AI is a global priority. The STANDING Together recommendations are a major step forward in ensuring equity for AI in health."

Dominic Cushnan, Deputy Director for AI at NHS England said: "It is crucial that we have transparent and representative datasets to support the responsible and fair development and use of AI. The STANDING Together recommendations are highly timely as we leverage the exciting potential of AI tools and NHS AI Lab fully supports the adoption of their practice to mitigate AI bias."

The recommendations are available open access via The Lancet Digital Health.

These recommendations may be particularly helpful for regulatory agencies, health and care policy organisations, funding bodies, ethical review committees, universities, and government departments.

Alderman JE, Palmer J, Laws E, McCradden MD, Ordish J, Ghassemi M, Pfohl SR, Rostamzadeh N, Cole-Lewis H, Glocker B, Calvert M, Pollard TJ, Gill J, Gath J, Adebajo A, Beng J, Leung CH, Kuku S, Farmer LA, Matin RN, Mateen BA, McKay F, Heller K, Karthikesalingam A, Treanor D, Mackintosh M, Oakden-Rayner L, Pearson R, Manrai AK, Myles P, Kumuthini J, Kapacee Z, Sebire NJ, Nazer LH, Seah J, Akbari A, Berman L, Gichoya JW, Righetto L, Samuel D, Wasswa W, Charalambides M, Arora A, Pujari S, Summers C, Sapey E, Wilkinson S, Thakker V, Denniston A, Liu X.
Tackling algorithmic bias and promoting transparency in health datasets: the STANDING Together consensus recommendations.
Lancet Digit Health. 2024 Dec 12:S2589-7500(24)00224-3. doi: 10.1016/S2589-7500(24)00224-3

Most Popular Now

Unlocking the 10 Year Health Plan

The government's plan for the NHS is a huge document. Jane Stephenson, chief executive of SPARK TSL, argues the key to unlocking its digital ambitions is to consider what it...

Alcidion Grows Top Talent in the UK, wit…

Alcidion has today announced the addition of three new appointments to their UK-based team, with one internal promotion and two external recruits. Dr Paul Deffley has been announced as the...

AI can Find Cancer Pathologists Miss

Men assessed as healthy after a pathologist analyses their tissue sample may still have an early form of prostate cancer. Using AI, researchers at Uppsala University have been able to...

AI, Full Automation could Expand Artific…

Automated insulin delivery (AID) systems such as the UVA Health-developed artificial pancreas could help more type 1 diabetes patients if the devices become fully automated, according to a new review...

How AI could Speed the Development of RN…

Using artificial intelligence (AI), MIT researchers have come up with a new way to design nanoparticles that can more efficiently deliver RNA vaccines and other types of RNA therapies. After training...

MIT Researchers Use Generative AI to Des…

With help from artificial intelligence, MIT researchers have designed novel antibiotics that can combat two hard-to-treat infections: drug-resistant Neisseria gonorrhoeae and multi-drug-resistant Staphylococcus aureus (MRSA). Using generative AI algorithms, the research...

AI Hybrid Strategy Improves Mammogram In…

A hybrid reading strategy for screening mammography, developed by Dutch researchers and deployed retrospectively to more than 40,000 exams, reduced radiologist workload by 38% without changing recall or cancer detection...

New Training Year Starts at Siemens Heal…

In September, 197 school graduates will start their vocational training or dual studies in Germany at Siemens Healthineers. 117 apprentices and 80 dual students will begin their careers at Siemens...

Penn Developed AI Tools and Datasets Hel…

Doctors treating kidney disease have long depended on trial-and-error to find the best therapies for individual patients. Now, new artificial intelligence (AI) tools developed by researchers in the Perelman School...

Are You Eligible for a Clinical Trial? C…

A new study in the academic journal Machine Learning: Health discovers that ChatGPT can accelerate patient screening for clinical trials, showing promise in reducing delays and improving trial success rates. Researchers...

New AI Tool Addresses Accuracy and Fairn…

A team of researchers at the Icahn School of Medicine at Mount Sinai has developed a new method to identify and reduce biases in datasets used to train machine-learning algorithms...

Global Study Reveals How Patients View M…

How physicians feel about artificial intelligence (AI) in medicine has been studied many times. But what do patients think? A team led by researchers at the Technical University of Munich...