Category Archives: Journal Club

Musings form the evidence based journal club at St Emlyns

Croup: Riding the Dex Express

Sooooo….this paper turned up at JC last week (thanks to Nicola P) and whilst I’m not sure that it meets all three of our criteria for a top JC paper it is relevant as a week barely seems to go by without someone questioning the dose/route/brand/colour/size/ethnicity of medicine for croup.
Rule of thumb chaps and chapesses ‘The greater the dogma, the greater the ignorance’. Someone cleverer than I said that, but I’m happy to plagiarise ‘cos it’s true.
Anyway, Croup arrives as a question once again in the journal Emergency Medicine Australia, but this time the question relates to speed of onset in mild to moderate croup.

 STOP! If you are in exam mode at this point you should read the paper. See what you think about it and give it a mark out of 10.

We’ve talked about this paper and it’s a tricky one. The first question is why has this paper been done (which we cannot answer, but can surmise privately). The use of steroids in the management of croup is very well established and is something we led on here in Virchester many years ago. It was even one of the very first BETs back in 2004 (amazing to think that we are still talking about this 8 years later).
I’ve also seen the Cochrane review and even examined some CTRs for FCEM on the subject. So, it pretty much seems to me that the question of whether we give steroids for croup is well made. The research that remains is, I suppose, about refining and polishing what is surely a well established fact.

STEROIDS WORK IN CROUP Click the link and read the Cochrane review.

So, what about the paper this week? Is there anything we can draw from it and learn? Well, the authors have done an RCT (good) on mild/moderate croup patients. Interesting this as for the mild ones would you give steroids or just let nature take its course? (Ed – depends on how mild as croup score 1-3 is mild) I’m not sure so there maybe an element of over-treatment in comparison to other practices. Whatever, the authors tell us that there is an effect of giving steroids that they can define and detect at  30 minutes following administration of steroid and that this counteracts the information given through Cochrane about a delayed effect taking up to 6 hours.

I have major concerns with this paper and I just don’t see how this is going to make a significant difference to our practice in PEM.  I don’t think a paper like this would appear in an exam, but if it did I would be pulling holes in it along the following lines.

1. What is the clinically important question here? It seems that we are looking to see the speed of onset of steroid meds in mild/moderate croup. The clinical importance of this is perhaps unclear except in logistical (admission) terms. What defines a significant difference in this low acuity group? Mild croup is not admitted anyway so what is the issue we are addressing?

2. Sample size. OK. An interest of mine, and if you share that interest (you sad person) then hop over to the podcast to hear more about how to understand and interpret sample size calculations. In this paper they appear to be using tests for continuous data for data which is unlikely to be so. Honestly, it seems as though these are the wrong tests for this data, but there is insufficient information in the paper for us to tell. Where is the clue? Well, the Wesley croup score is a categorical score (at best ordinal). It’s not continuous and is unlikely to be normally distributed, so a t-test is rarely going to be the right test. So hmm, not enough information to know but questions are there to be asked. If you want to know more about stats for Critical Appraisal then click here and here. Apart from anything else, a study of just 70 patients would have to show a massive effect if it is be valid and I don’t see that here. Similarly the graph shows average scores only, and I’m not sure that I’m just interested in the change in average score amongst 35 patients. I want to see the distribution as well. This is a common problem in papers as the mean score reporting removes the depth and character of the data.

3. Right, so we are unsure of the validity of the question and also of the sample size what else? Well,  do applaud the authors for defining the numbers of patients that they ‘could’ have recruited and the difference between that number (828) and the number recruited (70) is huge. This suggests a degree of patient selection which may well affect the results. Now, I don’t want to put a massive downer on this as it is an inevitable problem with EM research, but this ratio really asks questions as to whether this is a representative sample, or whether the results will be heavily skewed because it is a sample of convenience.

So, it sounds as though we were pretty down on this paper from a methodological point of view. We gave it a 3/10 to be honest which is clearly not high, but just wait is there ANYTHING we can take away from this piece of work at all. Well, it’s tricky to be honest. It’s likely (but I’m finding it difficult to tell) that oral dex starts working fairly quickly, but that was never a clinical dilemma for me before I read this paper so I’m not going to change practice. However, it’s a useful to use this as a vehicle to discuss Croup (again), to review the relevant BETs and to talk about how to spot flaws in papers.

 bw

Simon C

PS. If you are still in exam mode try answering the following questions…

1. What is meant by the term ‘double-blinded’ and why is it important in a trial like this?

2. Four patients in the placebo group worsened during the initial phase of the trial and were then given steroids. They were analysed in the placebo group despite getting steroids. What is this type of analysis called and is it the right approach?

Journals are dead: Long live the Journal Club

“The report of my death was an exaggeration”

Mark Twain

Just a quicky and a link out to our guide on Emergency Medicine Journal Clubs. Despite the rumours of the imminent demise of all medical journals, we at @stemlyns strongly believe that this will not lead to the death of journal clubs. Even if paper publication wanes (and it probably will) it will be even more important for clinicians to have the skills and abilities to be wary of what ‘evidence’ is out there.

For example anyone can now set up a Blog (Er, not sure that’s the right message here – Ed) and say what they like. How do you know it’s fair comment and good enough to change practice?

You do need, and you will always need to be a sceptic with the skills to critically appraise and critique the evidence and we think that a Journal Club is a great way to learn.

Read more here on our top tips for making your Emergency Medicine Journal Club effective, productive and worthwhile.

Our Journal Club runs on a Friday lunchtime in the ED. We’ll review, debate and then blog on the papers we discuss. Watch the blog for the latest in EM Critical Appraisal.

vb

Simon C

PS: We’ll be keeping a log of papers reviewed in our Journal Club from now on. If it works then we should have a rolling program of the best, most current and most relevant papers for Emergency Medicine. If you’re coming up to an exam….it’s a good place to visit.

ADAPT my management of low risk chest pain? I’m not sure I will…….

Image

Another Friday another JC in Virchester. Lots of chin stroking and pontification on this one.

We looked at the ADAPT trial by Martin Than and colleagues. Essentially they assessed the diagnostic utility of an accelerated protocol using Timi risk score, 0 & 2 hour troponin I measurements for rapid exclusion of cardiac pathology in patients attending the ED with chest pain. I was rubbing my hands on this one. “think of the bed days we can save on the decision unit. Say goodbye to all the 12 hour boredom of the ‘rule out’ MI admission.”

So what did they actually do? Well, over 2 sites in Oz and NZ they assessed 1975 patients attending with chest pain of suspected cardiac origin. They performed baseline and 2 hour Troponin I measurements that were unavailable to clinicians. They performed usual diagnostic protocols and followed patients up for 3 months. Then an adjudication panel apportioned the diagnosis of Major Adverse Cardiac Events at the end of the study period. They subsequently looked back to ask 2 main questions:

Firstly, what was the sensitivity of their accelerated diagnostic protocol (did it catch all the true positives)?

Secondly, if we assume the ADP is highly sensitive and therefore ‘safe’ how many patients would have been potentially dischargeable directly from the ED within 4 hours?

The methodology was pretty sound. Pretest probability for MACE was a little on the low side (15.3%) compared to our prevalence (20%) or other studies (30%). But their use of MACE is in keeping with much of the literature on the topic. The adjudication committee was not independent but we had a long discussion about this and concluded that actually, if they were blind to the Trop I assay, they couldn’t introduce much bias here even if they wanted to. If they underestimated MACE based on some of the more subjective aspects for example, they could potentially heighten the sensitivity of the diagnostic protocol – but this would worsen specificity and consequent diagnostic utility. Thus they would be shooting themselves in the foot.

And what did they find?

An excellent sensitivity of 99.7% (95% CI 98.1 – 99.9)

A dischargeable proportion of 20% using the ADP, with 1 MACE event in this group only (0.25%).

These are very interesting findings from a large, methodologically robust diagnostic study from 2 countries with a distinguished author list.  So, are we going to start applying the findings in practice?  I’m not so sure.  The findings must be taken in context with other concerns about the paper. We only had a few issues, but here they are:

  1. There was no mention of symptom duration at presentation in the cohort. Thus we don’t know how long these patients had had symptoms for prior to presentation. This is important: if most of the 1975 presented at 10 hours then actually, your 2 hour Troponin is predominately a 12 hour Troponin if you use time from symptom onset for your exclusion. We do Troponins 12 hours after symptom onset, but other systems (for example in many American papers) a measure of 12 hours after ED presentation is used.
  2. Their findings have not yet been externally validated. The sensitivity especially is likely to be slightly overestimated in a tightly controlled exploratory cohort. Something to be cautious of.
  3. They used Trop I rather than one of the new all reaching brand new spanking best in show highly sensitive troponin assays.  This might mean that the findings are more immediately relevant if you’re not using a high sensitivity assay, but for us in Europe this is a concern.  The proportion of patients who could be discharged is likely to be lower with a high sensitivity assay.
  4. The proportion of dischargeable patients using this rule was actually not huge at 20%. In the UK, this may be even less (only 7.4% of our patients had a TIMI risk score of 0/7 in previous work).  It follows that the others all end up admitted to some sort of inpatient service. This is a fairly self fulfilling prophecy in terms of investigations and therapeutics, as noted by their figures of 74% ADP negative patients receiving further investigations and 14% recieving therapy of some form.  There are risks with this.  And remember, the actual prevalence is only 15.3% (increased to 18-19% if we discount the ADP positive patients). 60% are still getting a raw deal (full inpatient investigation and treatment with no clear diagnosis at the end) Are we potentially therefore doing more harm by using a strict diagnostic protocol such as this one that will no doubt encourage admission due to the limited specificity?

The accompanying editorial to this paper can be found here and is a really nice summary of how far we have come and how much work still needs to be done on this topic. It also highlights the recent work looking at undetectable HsTnT to exclude MI at the door by a certain Dr Body (who?) and Professor Carley (never heard of him). This kind of work interests me a little more if I am honest.

I want a test which I can use immediately on the young patient with low risk chest pain that will rapidly reassure me about an absence of cardiac pathology. I am happy that Troponin is not this test in the elderly and will often manage them as ACS irrespective of initial trop results.

But if you can find me a way of getting all those 30-40 year olds who wait 12 hours only to be told that they haven’t had a heart attack (no s***) but that we’re not really sure what’s wrong with them,  off my decision ward, then I will applaud you.

Carry on the good work sirs.

dan

IST-3

So….

We have just covered the following paper in our journal club at Manchester Royal Infirmary. http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)60768-5/abstract

The benefits and harms of intravenous thrombolysis with recombinant tissue plasminogen activator within 6 h of acute ischaemic stroke (the third international stroke trial [IST-3]): a randomised controlled trial

Background

Thrombolysis is of net benefit in patients with acute ischaemic stroke, who are younger than 80 years of age and are treated within 4·5 h of onset. The third International Stroke Trial (IST-3) sought to determine whether a wider range of patients might benefit up to 6 h from stroke onset.

Methods

In this international, multicentre, randomised, open-treatment trial, patients were allocated to 0·9 mg/kg intravenous recombinant tissue plasminogen activator (rt-PA) or to control. The primary analysis was of the proportion of patients alive and independent, as defined by an Oxford Handicap Score (OHS) of 0—2 at 6 months. The study is registered, ISRCTN25765518.

Findings

3035 patients were enrolled by 156 hospitals in 12 countries. All of these patients were included in the analyses (1515 in the rt-PA group vs 1520 in the control group), of whom 1617 (53%) were older than 80 years of age. At 6 months, 554 (37%) patients in the rt-PA group versus 534 (35%) in the control group were alive and independent (OHS 0—2; adjusted odds ratio [OR] 1·13, 95% CI 0·95—1·35, p=0·181; a non-significant absolute increase of 14/1000, 95% CI −20 to 48). An ordinal analysis showed a significant shift in OHS scores; common OR 1·27 (95% CI 1·10—1·47, p=0·001). Fatal or non-fatal symptomatic intracranial haemorrhage within 7 days occurred in 104 (7%) patients in the rt-PA group versus 16 (1%) in the control group (adjusted OR 6·94, 95% CI 4·07—11·8; absolute excess 58/1000, 95% CI 44—72). More deaths occurred within 7 days in the rt-PA group (163 [11%]) than in the control group (107 [7%], adjusted OR 1·60, 95% CI 1·22—2·08, p=0·001; absolute increase 37/1000, 95% CI 17—57), but between 7 days and 6 months there were fewer deaths in the rt-PA group than in the control group, so that by 6 months, similar numbers, in total, had died (408 [27%] in the rt-PA group vs 407 [27%] in the control group).

Interpretation

For the types of patient recruited in IST-3, despite the early hazards, thrombolysis within 6 h improved functional outcome. Benefit did not seem to be diminished in elderly patients.
________________________________________________________________________________
 
 
So what did we think? Well, a lot of colleagues had already raised questions about it (@DrGDH has an excellent article on his blog here). It was very interesting to chew the fat over this paper for an hour.

Statistical gimmickry, ordinal analyses, logistic regression and a triple reducing power calculation aside, let’s look at the headline stats.

An 4% Absolute risk increase of death in the T’lysis group within the first 7 days.

An overall ARR of 2% for the primary outcome of alive with Oxford Handicap Score ❤ at 6/12 with T’lysis

An overall mortality rate of 27% in both control and T’lysis groups at 6/12.

Thus, we can take things back to basics……let’s think through an example of how this might work in practice.

What if 100 people aged 50-80 with varying severities of stroke present to my emergency department with a time onset of anywhere between 0 and 5 hours.

I thrombolyse 50 of them. The others get standard care.

In the Thrombolysed group, 2 of the 50 will be dead within a week. 1 more than the control arm will be alive and relatively independent at 6 months.

In both groups, about 13-14 will be dead in 6/12 regardless.

Not the kind of figures that make me want to redirect ambulances and develop a regional stroke network. And not the kind of pharmacotherapy I would want for my 90 year old nan. Goes with the previously maligned views published on the best bets website

Any dissenters???  Who is up for it based on face validity and the possibility of type 2 error in this trial? Who has a golden anecdotal story? Who thinks the benefits outweigh the risks of being a wheelchair bound dysphagic dependent?

Time for a serious contemplation of why we invest so heavily in therapies that have such potential to harm.

Dan.

The Critical Appraisal FCEM exam

If you are planning on being an emergency physician in the UK then you will probably be thinking about trying to get through the FCEM exam. At the current time that means that you are going to have to pass the critical appraisal component. This is a 90 minute written exam which you have to pass if you want to get those magical post nominals and your ticket to a consultant post. But…..

…….lots of people fail the critical appraisal exam. Why?

Well, because they don’t answer the questions obviously, but it’s perhaps a bit more complex than that as having seen some of the answers and having done loads of practice sessions over the last few years there are common errors that candidates make. So, here are a few tips on how to get through the exam, these are my thoughts alone and they do not represent anyone else (and especially not the College).

1. Read the online guide to the critical appraisal. Have a look at past questions and common errors. Notice how often the same errors seem to get made in each round of exams!

2. Practice, practice, practice. If you don’t read papers on a regular basis then start now. You really need to do this in a group so if you don’t have a journal club start one. There are loads of models out there, choose something that works for you. This link is our original model from 1998!!! which is still not a bad way to start.

3. Don’t try and be clever. Lots of people want to use big words and technical terms when the question doesn’t ask for it. If  you get asked to explain something e.g. blinding, then explain it. Don’t try and list 15 sorts of Bias that blinding hopes to avoid. Just answer the question in terms that you would use to a colleague (the examiners are your future colleagues so explain in those terms).

4. Look at the size of the area on the sheet given for the answer. If the answer area on the form is a small box the clue is that the examiners are looking for a short answer. If you find yourself writing outside of the allocated space you are either a – writing with a crayon or b – missing the point of the question.

5. A major part of the exam is writing an abstract which will have been blanked out. This can be tricky and it’s easy to get waylaid. My advice is to use the standard Objectives, Methods, Results, Conclusions format. Make sure these link up! Crazy people have an objective that does not match the results, which is a separate point to the conclusion. Don’t do that. Start with the Aim as stated by the authors. This will give you the objective AND tell you what you should be looking for in terms of the principal finding for the results, that then leads nicely into giving you a conclusion.

6. There will only be two sorts of papers. Theraputic (prob an RCT) or diagnosis. Focus your practice and learning on these types of papers.

7. Learn some basic stats terms that you can explain to a colleague. You don’t need to know hard stuff but the basics should be there. If you want a basic guide to stats for crit app then you can listen to a few old stats lectures we put together specifically for people aiming at getting through a critical appraisal test. Stats 1 ; Stats 2 They are bit old and were early attempts, but if you find them useful that’s great.

8. Don’t talk about stuff you don’t understand. Randomly inserting words that you think might be relevant without understanding them rarely helps.

9. Write clearly. Candidates in a VIVA can explain if something is unclear. In a written exam you have to be able to read and interpret what is on paper. Make sure it makes sense.

10. Write like a pharmacist, not like a doctor. If it cannot be read, it cannot be marked. Similarly one quote I do like from recent feedback is self explanatory

    • Many candidates wrote correct statements – but they were not relevant to the answer…..

So. It’s not a difficult exam if you already do critical appraisal as part of your practice and if you regularly read papers. It’s an easy exam to pass, but also easy to fail if you try and be too clever or technical.