The NIMH issued a press release about the publication in JAMA Psychiatry of results of the ED-SAFE Study, the largest suicide intervention trial ever conducted in emergency departments (ED) in US.
NIMH
“We expect that EDs are capable of helping individuals at risk for suicide attempts. Earlier ED-SAFE study findings showed that brief universal screening could improve detection of more individuals at risk,”, said Jane Pearson, Ph.D., chair of the Suicide Research Consortium at the NIMH. “These recent findings show that if ED care also includes further assessment, safety planning, and telephone-based support after discharge, there is a significant reduction in later suicide attempts among adults.”
“We were happy that we were able to find these results,” said lead author Ivan Miller, Ph.D., Professor of Psychiatry and Human Behavior at Brown University, Providence, Rhode Island. “We would like to have had an even stronger effect, but the fact that we were able to impact attempts with this population and with a relatively limited intervention is encouraging.”
The report of the study in JAMA Psychiatry
Miller IW, Camargo CA, Arias SA, Sullivan AF, Allen MH, Goldstein AB, Manton AP, Espinola JA, Jones R, Hasegawa K, Boudreaux ED. Suicide prevention in an emergency department population: the ED-SAFE Study. JAMA Psychiatry. 2017 Apr 29.
The recently revamped website for the JAMA network of journals provided updated reports of the heavy traffic being drawn in by the article.
The new Key Points feature for important articles gave succinct, more quickly digestible summary of the study than the similarly spun abstract.
Key Points
Question Do emergency department (ED)–initiated interventions reduce subsequent suicidal behavior among a sample of high-risk ED patients?
Findings In this multicenter study of 1376 ED patients with recent suicide attempts or ideation, compared with treatment as usual, an intervention consisting of secondary suicide risk screening by the ED physician, discharge resources, and post-ED telephone calls focused on reducing suicide risk resulted in a 5% absolute decrease in the proportion of patients subsequently attempting suicide and a 30% decrease in the total number of suicide attempts over a 52-week follow-up period.
Meaning For ED patients at risk for suicide, a multifaceted intervention can reduce future suicidal behavior.
The abstract elaborates:
Results A total of 1376 participants were recruited, including 769 females (55.9%) with a median (interquartile range) age of 37 (26-47) years. A total of 288 participants (20.9%) made at least 1 suicide attempt, and there were 548 total suicide attempts among participants. There were no significant differences in risk reduction between the TAU and screening phases (23% vs 22%, respectively). However, compared with the TAU phase, patients in the intervention phase showed a 5% absolute reduction in suicide attempt risk (23% vs 18%), with a relative risk reduction of 20%. Participants in the intervention phase had 30% fewer total suicide attempts than participants in the TAU phase. Negative binomial regression analysis indicated that the participants in the intervention phase had significantly fewer total suicide attempts than participants in the TAU phase (incidence rate ratio, 0.72; 95% CI, 0.52-1.00; P = .05) but no differences between the TAU and screening phases (incidence rate ratio, 1.00; 95% CI, 0.71-1.41; P = .99).
I have the benefit of having read the entire article a number of times, but there are some notable statistics being reported in the abstract and some crucial things being left out.
The phase of the study that involved only introducing screening into treatment as usual (TAU) had no effect on suicide attempts (p= .99). The claim of an effect of the more extensive intervention on suicide attempts depends on multivariate analyses that include a confidence interval that includes 1.0. (incidence rate ratio, 0.72; 95% CI, 0.52-1.00; P = .05).
From JAMA Psychiatry
Results are quite weak, at best. Pairwise comparisons are being reporting, first the screening versus TAU, then the more extensive intervention versus TAU. Missing is any reporting of the overall ANOVA testing whether there is at least one significant pairwise difference between groups. Obtaining such a significant difference would justify a post hoc look at the specific pairs. Given what we have already been told in the abstract, it is safe to assume no overall effect. This is a null trial. If we stuck to a priori statistical plans, we would have to say that a phased-in, comprehensive intervention with suicidal patients presenting in an emergency room failed to impact subsequent suicide attempts.
These findings contradict the statement of the NIMH Chair of the Suicide Research Consortium.
I know, it is arbitrary to make go/no go decisions based on an arbitrary level of significance, p< .05 or whatever. Yet, the implement/don’t implement and evidence-supported/not evidence-supported distinctions are binary. The best we can do is to set criteria based on a power analysis and avoid switching criteria when we don’t obtain the results that we would have liked.
We can stop here in our critique with the usual messages to avoid spinning of results in order to obtain politically expedient and socially satisfying, even if inaccurate conclusions. Once again, results of a trial are being exaggerated to justify a conclusion to which the researchers and policy makers are already committed.
But there is a lot more to be learned from this report of a large and historically significant trial.
Who was enrolled and what treatments were offered?
1376 adult participants were selected from persons presenting to 8 emergency departments across 7 states with participants with a suicide attempt or ideation within the week prior to the ED visit. Patients under 18 were excluded.
In the TAU phase, participants were treated according to the usual and customary care at each site, serving as the control for the subsequent study phases.
In the screening phase, sites implemented clinical protocols with universal suicide risk screening (the Patient Safety Screener) for all ED patients.
In the intervention phase, in addition to universal screening, all sites implemented a 3-component intervention: (1) a secondary suicide risk screening designed for ED physicians to evaluate suicide risk following an initial positive screen, (2) the provision of a self-administered safety plan and information to patients by nursing staff, and (3) a series of telephone calls to the participant, with the optional involvement of their significant other (SO), for 52 weeks following the index ED visit.
The outcome
The outcome was the proportion of patients who made a suicide attempt and the total number of suicide attempts occurring during the 52-week follow-up period.
Overall, of 1376 participants, 288 (20.9%) made at least 1 suicide attempt during the 12-month period. In the TAU phase, 114 of 497 participants (22.9%) made a suicide attempt, compared with 81 of 377 participants (21.5%) in the screening phase and 92 of 502 participants (18.3%) in the intervention phase. Five attempts were fatal, with fatalities observed in the TAU phase (n = 2) and intervention phase (n = 3).
Suicide attempts can be interpreted as an outcome in itself or as a surrogate outcome for deaths by suicide. Despite the substantial sample size, there is no way that this study could have demonstrated a significant reduction in deaths by suicide. That reflects the infrequency of death by suicide, even in such a high risk population. The ratio of 57.6 suicide attempts per one death by suicide is much higher than what is typically observed (usually in the range of 100 or so per suicide. This probably reflects the high risk nature of this population, as well as the methodology for determining the serious of suicide attempts.
More evidence that screening for suicide doesn’t improve outcomes
This study adds to an accumulation of a lack of evidence that routine screening for suicide is either efficient or leads to less suicides.
Previously, I blogged about the SEYLE trial of a school-based intervention to prevent teen suicide. It was a large RCT, but failed to demonstrate that screening affected the likelihood of a suicide attempt. The null findings for the Screening by Professionals programme (ProfScreen) of SEYLE are generally downplayed.
Another blog post Use of scales to assess risk for a suicide attempt wastes valuable clinical resources discussed a large UK study that found none of the commonly used screening scales were clinically useful in predicting subsequent suicide.
That study concluded
Risk scales following self-harm have limited clinical utility and may waste valuable resources. Most scales performed no better than clinician or patient ratings of risk. Some performed considerably worse. Positive predictive values were modest. In line with national guidelines, risk scales should not be used to determine patient management or predict self-harm.
Nonetheless there is:
The Joint Commission. Detecting and treating suicide ideation in all settings. Sentinel Event Alert. 2016;(56):1-7.
The Joint Commission is a United States-based nonprofit tax-exempt 501(c) organization[1] that accredits more than 21,000 health care organizations and programs in the United States. The Joint Commission recommends that hospitals routinely screen patients for risk of suicide.
An editorial accompanying the JAMA Psychiatry report cited this recommendation as part of the rationale of the ED-SAFE study and warned of implementing screening without resources:
Since the alert, many hospitals have implemented suicide risk screening without the benefit of evidence-based tools and clinical pathways, potentially increasing the risk of underdetection (ie, false-negatives) or overburdening limited mental health resources with false-positives.
Most patients in the ED-SAFE study were not recorded as receiving the intervention as intended.
Medical record review indicated that 449 of 502 participants (89.4%) had received a suicide risk assessment from their physician, but only 17 (3.9%) had documentation of the ED-SAFE standardized secondary screening was used.
And
Among those participants who completed the initial CLASP call, 114 (37.4%) reported having received a written safety plan in the ED.
You cannot fault these researchers for having failed to make a concerted effort to train personnel in the participating sites or to systematically implement the study protocol. See
Boudreaux ED, Camargo CA, Arias SA, Sullivan AF, Allen MH, Goldstein AB, Manton AP, Espinola JA, Miller IW. Improving suicide risk screening and detection in the emergency department. American Journal of Preventive Medicine. 2016 Apr 30;50(4):445-53.
A wealth of evidence suggests that is it is difficult to implement formal screening with self-report and interviewer-completed checklists in medical settings. Most medical personnel find such instruments intrusive and they are not efficient, anyway. Alex Mitchell and I documented this in our book, Screening for Depression in Clinical Practice: An Evidence-Based Guide
.In both the screening and intervention phase, it was difficult to get adherence to the protocol, in part because patients entering EDs are not necessarily cooperative. But more importantly, EDs in this study were not well-connected to the specialty mental health services needed for timely follow up. The accompanying editorial notes:
Although EDs have been conceptualized as key sites to identify and treat individuals at high risk for suicide,8 the troubling reality is that mental health resources are not available in most American EDs, and few universally screen for suicide risk.9,10 Notably, participating ED-SAFE study sites did not have psychiatric services within or adjacent to the ED in order to increase generalizability. Although time constraints, inadequate training, and lack of proper screening instruments have been cited as reasons clinicians do not routinely screen for suicide risk,8,10,11 the absence of psychiatric services in most EDs reflects disproportionately low cultural expectations of the ED in addressing potentially life-threatening mental health crises.
The realignment and reallocation of resources needed to address this practical and structural problem are not easily obtained. Clinical instances in which quick referral and follow up of a seriously suicidal patient are relatively infrequent. It is difficult to maintain the personnel and resources unencumbered until they are needed, especially in the face of other, pressing competing demands.
How will ED-SAFE be cited and entered into the accumulating literature concerning the difficulty getting reductions in lives lost to suicide?
The article reports the Number Needed to Treat (NNT) for patients receiving the comprehensive ED-SAFE intervention:
The NNT to prevent future suicidal behavior ranged between 13 and 22. This level of risk reduction compares favorably with other interventions to prevent major health issues, including statins to prevent heart attack (NNT = 104),23 antiplatelet therapy for acute ischemic stroke (NNT = 143),24 and vaccines to prevent influenza in elderly individuals (NNT = 20).25
But if the intervention is not effective, NNTs are misleading.
If the NIMH press release is taken as a sign, the ED-SAFE intervention will be interpreted as impressively effective. However, despite some spinning, the ED-SAFE researchers present the problems they encountered and the results they obtained in a way that the formidable obstacles to such a well-conceived effort succeeding are apparent. It would be unfortunate if the lessons to be learned are missed.
