Predictive models in perioperative medicine

Identifying and mitigating biases in perioperative prognostic models and clinical scoring systems

Predictive models are widely used by clinicians working in perioperative care to assess patients who are waiting for surgery. These models are useful because they simplify complex problems, however, they are not 100% accurate.

If a model gives the wrong answer, patients may be misdiagnosed or offered the wrong treatment. This might mean that patients who need surgery are not offered it, or that surgery is offered when it is not helpful. If models are wrong more often for particular groups, this could worsen health disparities.

This project aims to help healthcare professionals working in perioperative medicine choose the model which is best for a particular patient, and to ensure that future models help as many patients as possible.
Digital health innovations have the potential to dramatically alter the way healthcare is delivered in the UK, improving access to cutting edge therapies and freeing up clinicians to focus on tasks humans do best. However, they can also exacerbate existing health inequity, and may become a new source of inequity, systematically disadvantaging certain groups in society.

Predictive models and clinical scores (PMCS) use health data to make predictions & guide clinicians’ decision-making in diagnosis, treatment planning, prognosis, and other parts of the patient care pathway. Since the 1980s, thousands of PMCS have been developed, and their use has become ubiquitous throughout healthcare. Some PMCS assist clinicians make key decisions about whether to recommend entry to care pathways, for instance by estimating the risks associated with surgery for a particular patient. Other PMCS predict potential for clinical deterioration, and may partially govern referral to critical care services.

Clinicians may expect that PMCS perform well for all patients, but we often lack studies to confirm this. A well-known surgical risk prediction score created in the UK had poor ‘calibration’ when tested in a New Zealand cohort, causing it to under-predict risk for patients. A recent evidence review by the National Institute for health and Care Excellence (NICE) highlighted similar issues with calibration for other PMCS predicting perioperative risk, though it did not comment on demographic subgroup performances. Given scores’ unpredictable performance at population level, it cannot be assumed that their performance & calibration is equitable across subgroups within populations.

In order to identify areas where PMCS may adversely impact patient care we need to understand:
a) Which PMCS are most commonly used, and
b) How these influence clinicians’ decision making.

A cross-sectional survey will be conducted, asking perioperative healthcare professionals (doctors, nurses and allied health professionals) which types of PMCS they use, and in what contexts. Respondents will also be asked about their perceptions of the potential for bias in PMCS. A focus group involving patients and healthcare professionals will explore which PMCS carry the greatest risk of contributing to harm, including health inequity. Semi structured interviews will be used to explore how doctors select and use scores, and how they perceive scores influence their decision making. Finally, an extensive quantitative analysis will externally validate a subset of PMCS, to evaluate how well they perform, and to investigate the impact of any differences in performance across sociodemographic subgroups.

Dr Joseph Alderman (University of Birmingham & University Hospitals Birmingham) Dr Dhruv Parekh (University of Birmingham & University Hospitals Birmingham) Prof Charlotte Summers (University of Cambridge & Cambridge University Hospitals NHS Foundation Trust) Prof Richard Riley (University of Birmingham) Dr Xiaoxuan Liu (University of Birmingham & University Hospitals Birmingham) Prof Alastair Denniston (University of Birmingham & University Hospitals Birmingham)

Project Team

Project Details

Start Date: 01/08/2023

End Date: 04/02/2025

PATHWAY – UHB Health Data Research Hub

HDR Midlands Project

West Midlands Secure Data Environment

HDR Midlands Project

RECORDER – Researching Rare Diseases

HDR Midlands Project