A simple truth applicable to COVID-19

One of my favorite lessons in statistics: Scott Braithwaite at JAMA Network.

SB is on target.
------------------------------------------
The six most dangerous words in evidence-based medicine (EBM) do not directly cause deaths or adverse events. They do not directly cause medical errors or diminutions in quality of care. However, they may indirectly cause these adverse consequences by leading to false inferences for decision making. Consider the following statements, each of which includes the six most dangerous words:

• There is no evidence to suggest that hospitalizing compared with not hospitalizing patients with acute shortness of breath reduces mortality.

• There is no evidence to suggest that ambulances compared to taxis to transport people with acute GI bleeds reduces prehospital deaths.

• There is no evidence to suggest that looking both ways before crossing a street compared to not looking both ways reduces pedestrian fatalities.

All of these statements are clearly absurd as foundations for decision making, yet they are technically correct. In each case, these hypotheses have been untested and therefore there is no evidence to suggest otherwise, presuming a definition of “evidence” that requires formal hypothesis testing in an adequately powered study.1 Indeed, as of this writing, “there is no evidence to suggest” appears in MEDLINE 3055 times, nearly as often as “decision analysis” (3140 times), a common framework for using evidence to make decisions. My anecdotal experience suggests that “there is no evidence to suggest” is a mantra for EBM practitioners, in a wide variety of settings. And it is infrequently followed by the clarifying aphorism “absence of evidence is not evidence of absence”2 or discussions of more inclusive definitions of “evidence.”3,4

Deciding not to intervene when “there is no evidence to suggest” the favorability of an intervention makes sense from a decision analytic perspective when the act involves potential harm or large resource commitments.5 However, deciding to intervene when “there is no evidence to suggest” also may make sense, particularly if the intervention does not involve harm or large resource commitments, and especially if benefit is suggested by subjective experience (eg, the qualitative analogue of the Bayesian prior probability).6

Indeed, the fundamental problem with the phrase “there is no evidence to suggest” is that it is ambiguous while seeming precise. For example, it does not distinguish between the vastly different evidentiary bases of US Preventive Services Task Force (USPSTF) grades I, D, or C, each of which may have distinct implications for decision making.7 “There is no evidence to suggest” may mean “this has been proven to have no benefit” (corresponding to USPSTF grade D), which has very different implications than alternative meanings for “there is no evidence to suggest” such as “scientific evidence is inconclusive or insufficient” (corresponding to USPSTF grade I) or “this is a close call, with risks exceeding benefits for some patients but not for others” (corresponding to USPSTF grade C). As a result, these six dangerous words may mask the uncertainty of experts. They even may be used to deny treatments with potential benefit, if they are interpreted as the equivalent of USPSTF grade D (“this has been proven to have no benefit”) but really mean the equivalent of USPSTF grade I (“scientific evidence is inconclusive or insufficient”).

Beyond its ambiguity, “there is no evidence to suggest” creates an artificial frame for the subsequent decision. It may signal to patients, physicians, and other stakeholders that they need to ignore intuition in favor of expertise, and to suppress their cumulative body of conscious experience and unconscious heuristics in favor of objective certainty. Suppressing intuition may be appropriate when the evidence yields robust inferences for decision making, but is inappropriate when the evidence does not yield robust inferences for decision making. Yet “there is no evidence to suggest” is compatible with either scenario. Because decisions are particularly sensitive to patient preferences when the favorability of an intervention is unclear (eg, USPSTF grade C), “there is no evidence to suggest” may inhibit shared decision making and may even be corrosive to patient-centered care.8 Indeed, it is instructive to note that most people make patient-centered decisions every day without high-quality (eg, randomized controlled trial) evidence, and these decisions are not always wrong. Furthermore, foundational papers in the EBM field make it explicitly clear that EBM was never meant to exclude information derived from experience and intuition.4 While some may argue that misuse of this phrase is only a symptom of not having received appropriate training in EBM, my experience with practitioners of EBM across the clinical, educational, research, and policy spectra suggests the contrary.

I suggest that academic physicians and EBM practitioners make a concerted effort to banish this phrase from their professional vocabularies. Instead, they could substitute one of the following 4 phrases, each of which has clearer implications for decision making: (1) “scientific evidence is inconclusive, and we don’t know what is best” (corresponding to USPSTF grade I with uninformative Bayesian prior) or (2) “scientific evidence is inconclusive, but my experience or other knowledge suggests ‘X’” (corresponding to USPSTF grade I with informative Bayesian prior suggesting “X”), (3) “this has been proven to have no benefit (corresponding USPSTF grade D), or (4) “this is a close call, with risks exceeding benefits for some patients but not for others” (corresponding to USPSTF grade C). Each of these four statements would lead to distinct inferences for decision making and could improve clarity of communication with patients.

EBM practitioners should abandon terms that may unintentionally mislead or inhibit patient-centered care. “There is no evidence to suggest” is a persistent culprit. Informed implementation of EBM requires clearly communicating the status of available evidence, rather than ducking behind the shield of six dangerous words.