Terrifying news from the UK: You are more likely to die if admitted to the hospital on the weekend than on a weekday. This can only mean one thing–patients admitted on weekends are being locked in and forced to battle to the death.
Kidding, kidding. Leaping to a single conclusion like that without evidence or consideration of multiple factors would be at best irresponsible and at worst dangerous.
Enter stage right: British Health Secretary Jeremy Hunt and the British Conservative Party, who have decided that the higher mortality rate among weekend admissions is due to reduced services on the weekends, which can only be solved by creating a “seven-day NHS.”
The problem is that this conclusion seems to have as much evidence to support it as my Battle Royale idea above.
Let’s look at this issue through the lens of today’s bad chart, from the Conservative Home article by Paul Abbott entitled “The graph that shows why Hunt is right about a Seven Day NHS”:
Y-axis label: Statistical risk of death compared with Wednesday baseline
X-axis label: Day of admission to hospital
Right away, before we even get into the flaws in the chart itself, we get a perfect illustration of the leap in logic that is at the heart of the Conservative argument. The graph does not show “why Hunt is right about a Seven Day NHS.” It doesn’t show he’s wrong, either. The graph does not give us any information one way or the other about a seven-day NHS.
Abbott is assuming correlation equals causation. The flawed logic goes like this: The chart shows an increased rate of death on weekends. Services are reduced on weekends. Therefore, reduced services are causing the increase.
But you can make any number of ridiculous claims using that logic alone, without any supporting evidence to show a causal link. For example: The chart shows an increased rate of death on weekends. Television programming is different on the weekends than on weekdays. Therefore, being stuck in the hospital with only crappy weekend programming to entertain/distract people is sapping their will to live.
You don’t even need to jump to the lethality of Murder She Wrote reruns to come up with any number of possible reasons for the increase in weekend hospital deaths. The first thing that popped into my mind was that more deaths would be expected among those admitted on weekends because people going in on the weekend are more likely to have an urgent or emergent problem. In the US, it’s more expensive to seek care over the weekend for most people, so seeking care over the weekend rather than waiting to make an appointment during the week often signals a problem that can’t wait to be addressed. Of course emergencies and urgent symptoms are going to be more likely to indicate a life-threatening problem.
The expense of weekend healthcare might not be an issue for patients in the UK, so maybe this conclusion is not accurate (although I can still see not wasting the weekend by going into the hospital for a non-urgent problem for those who work regular business hours).
Another possible explanation is that hospitals are less likely to admit patients who come in on the weekends unless the health problem is urgent, especially if services on the weekend are reduced–not BECAUSE of reduced services, mind you, but because the hospital is set up to provide routine care during the week, so that would be postponed, increasing the proportion of people with non-routine, higher risk problems among weekend admissions.
On a similar note, more weekday admissions are likely to be for routine, scheduled, and/or elective care, which would throw off the risk ratio when comparing weekends to weekdays. Some, all, or even none of these could directly affect the weekend admission death rate.
The point is that there are other possible reasons that alone or (more likely) together might explain the increase in deaths. Without evidence supporting a causal link, we can’t draw conclusions about the cause of the increase, much less attempt to make policy changes based on these conclusions, especially policy changes that are potentially expensive and have life-or-death consequences.
So Abbott’s graph does not show the conclusion he touts in the title.
Even the increases in deaths on Saturday and Sunday are exaggerated by starting the y-axis at 0.96 instead of 0. Here’s the same chart with a corrected y-axis:
EDIT 11/13/2015, 10am EST: Starting at 0 does not actually make sense for relative risk. A column chart is actually a misleading presentation for relative risk in the first place, so I’ve cut my own misleading corrected chart. Below is a more appropriate chart of the data, which is the source of Abbott’s numbers (see below). Thanks to Jamie Bernstein for pointing this out.
There is an increase, but is it significant enough to warrant the reaction from Hunt, Abbott, and others pushing for a seven-day NHS?
To find out, we have to go back to the data themselves. Abbott claims in his article that the graph is from the Journal of the Royal Society of Medicine. When asked for more details on Twitter, he provided this link, which shows a figure from Freeman et al., “Weekend hospitalization and additional risk of death: An analysis of inpatient data,” published in the Journal of the Royal Society of Medicine in 2012. The full text is available at that link, and the paper does not include Abbott’s graph, although the figure he links to appears to be the source of his data.
In fact, the authors of this paper recently provided an updated analysis using more recent numbers (2013-14), published in the BMJ under the title “Increased mortality associated with weekend hospital admission: a case for expanded seven day services?”
This analysis appears to be at the heart of the statements Hunt has been making since July about “11,000 excess deaths” annually in the UK, the alleged number of deaths above the Wednesday baseline that occur on weekends. Oh, and by the way, “weekend” now apparently includes Friday and Monday.
The subtitle of Freeman and colleagues’ analysis certainly suggests that they have found a causal link between these deaths and reduced services on weekends, yet the analysis itself does not study this link, and in the interpretation section, the authors write, “It is not possible to ascertain the extent to which these excess deaths may be preventable; to assume that they are avoidable would be rash and misleading.”
Hmmm, “rash and misleading,” kind of like the subtitle of the analysis? Oh, but they have a question mark at the end! It’s not a conclusion. They’re “just asking questions,” right? Sure, these questions suggest a conclusion and ignore the many, many other questions raised, but you can’t fault the researchers for conclusions other people draw from their innocent questions.
Seem familiar? I do believe we might be seeing the medical research equivalent of JAQing off.
Even if the difference in deaths is statistically significant and the methodology sound (which, judging from the criticism, is not a safe assumption), we don’t know if these deaths are even preventable, and the effects on the public of believing they risk death or poorer quality care on the weekends are themselves potentially harmful.
In fact, a group of physicians has started to document this, what they are calling the “Hunt effect,” the number of people who now avoid seeking care on the weekends out of fear created by Hunt’s misleading statements as well as the health outcomes for these people who delayed care. Whether they are able to demonstrate this effect as significant and conclusive remains to be seen, but it nevertheless illustrates that “just asking questions” and drawing causal conclusions based solely on correlation are potentially harmful even before any changes are implemented based on these flawed conclusions.
I personally think the deaths are worth looking into, even though it seems likely that even if there is a significant increase (still debatable), most are not preventable. But this is difficult to do when people have already made up their minds thanks to the success these politicians and scientists have had in trolling England.