Thanks for a great post, Gareth. If you’ve landed here without reading that post, hit the link – this is a follow on, a ‘deep dive’ in the words of Smart EM – to be taken in the context of Gareth’s main post.
This is a landmark study in Emergency Medicine and gives us something useful that could reduce admissions. With a critical appraisal hat on, however, I do think it’s important to point out a few flaws in the methodology.
Methodology of the Lancet study
The authors have essentially prospectively evaluated the performance of the GBS at several centres by reviewing case notes of patients presenting with upper GI bleed. They then prospectively implemented the GBS, discharging patients with GBS of 0 and found that it was safe and reduced admissions. It sounds pretty good, so why is there a problem?
The issue is that there’s no control group in the implementation phase. When clinicians are told to use a tool that enables discharge of low risk patients, they may decide to use it in particularly low risk patients, who they’re happy to consider early discharge for. There’s some evidence that this actually happened, as the proportion of low risk patients is greater in the implementation phase (22% vs. 16%) and the overall number of patients is enrolled is greater in the implementation phase (572 vs. 334), despite the overall recruitment period being shorter. This is the classic problem with simple before and after analyses, and it makes the comparison of admission rates before and after implementation subject to substantial bias.
What’s more, there’s the issue of resource utilisation. In the derivation phase, 96% of patients were admitted compared to 71% after implementation, which is great. However, the median length of stay didn’t change (2 days in each group) although the mean length of stay reduced. This suggests that the patients we’re avoiding admission for after implementation of the GBS would have had a short length of stay anyway (<2 days), so the reductions in length of stay are occurring in that group. That’s still OK – so far, we’re still on to a cost saving and patients get to go home earlier.
However, you also have to consider that the low risk patients who were discharged were all given outpatient endoscopies and outpatient follow-up. OK, only 40% actually attended for the endoscopy. But what we don’t know is how many of them would have undergone endoscopy and out-patient follow-up with standard care – it may well have been less than 40%. What’s more, using the score might tempt physicians to over-investigate or over-treat those who aren’t in the low risk group.
Overall impact on resource utilisation
It’s therefore possible that implementation of this protocol actually leads to a rebound overuse of resources. To get a better idea of whether this actually happens, we need a control group. The most obvious way to do that is to run an RCT. Patients could be individually randomised to care guided by the GBS or standard care, or we could use cluster randomisation (e.g. randomise each centre to deliver care guided by one intervention or the other). Alternatively, we could use a stepped wedge design, whereby we enrol a number of centres and all of them sign up to implementation of the GBS-based protocol. Each centre is given a randomly allocated implementation date. We then run a before and after analysis to evaluate admission rates and overall resource utilisation. This is still a before and after analysis, but we have contemporaneous control groups at different centres.
What’s a Service Evaluation?
There’s a final point to make here. The implementation phase was a service evaluation. What does this mean? Essentially, two centres implemented the protocol in practice and audited what happened. They didn’t get consent from patients. (They didn’t need it for this type of work). However, it does mean that they couldn’t actually follow patients up as they would in a research study. That means that the 60% of low risk patients who failed to show for their endoscopy went out into the ether. They could have attended other hospitals for further care, perhaps because they were disgusted at being inappropriately discharged! They may have undergone intervention at those centres – we just don’t know with this design.
The bottom line for clinical practice
Does this stop us from using the Glasgow Blatchford score? No, excepting a few methodological flaws I think these authors have, on the whole, shown its safety. I think we can use it. Even NICE says we can use it! We shouldn’t be so confident about the overall impact on resource utilisation though, as we just haven’t shown that in this study.