Comparison of PrognosticAccuracy of 3 Delirium Prediction Models

Hilde van Nieuw Amerongen, MSc, RN; Sandra Stapel, MD, PhD; Jan Jaap Spijkstra, MD, PhD; Dagmar Ouweneel, PhD, MSc; Jimmy Schenk, MSc, RN

Disclosures

Am J Crit Care. 2023;32(1):43-50.

Abstract and Introduction

Abstract

Background: Delirium is a severe complication in critical care patients. Accurate prediction could facilitate determination of which patients are at risk. In the past decade, several delirium prediction models have been developed.

Objectives: To compare the prognostic accuracy of the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, and to investigate the difference in prognostic accuracy of the PRE-DELIRIC model between patients receiving and patients not receiving mechanical ventilation.

Methods: This retrospective study involved adult patients admitted to the intensive care unit during a 2-year period. Delirium was assessed by using the Confusion Assessment Method for the Intensive Care Unit or any administered dose of haloperidol or quetiapine. Model discrimination was assessed by calculating the area under the receiver operating characteristic curve (AUC); values were compared using the DeLong test.

Results: The study enrolled 1353 patients. The AUC values were calculated as 0.716 (95% CI, 0.688–0.745), 0.681 (95% CI, 0.650–0.712), and 0.660 (95% CI, 0.629–0.691) for the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, respectively. The difference in model discrimination was statistically significant for comparison of the PRE-DELIRIC with the E-PRE-DELIRIC (AUC difference, 0.035; P = .02) and Lanzhou models (AUC difference, 0.056; P < .001). In the PRE-DELIRIC model, the AUC was 0.711 (95% CI, 0.680–0.743) for patients receiving mechanical ventilation and 0.664 (95% CI, 0.586–0.742) for those not receiving it (difference, 0.047; P = .27).

Conclusion: Statistically significant differences in prognostic accuracy were found between delirium prediction models. The PRE-DELIRIC model was the best-performing model and can be used in patients receiving or not receiving mechanical ventilation.

Introduction

Patients admitted to the intensive care unit (ICU) are at risk of severe complications developing. One of these complications is delirium, which is characterized by disturbances in attention, awareness, and cognition.^[1] A recent meta-analysis showed a mean incidence of delirium in the ICU of 29% (SD, 14%).^[2] Delirium during an ICU stay can have significant adverse consequences, including prolonged hospital stay, increased mortality risk, and higher cost of hospitalization.^[3–6]

Identifying patients with an increased risk of delirium may result in earlier diagnosis. Moreover, predisposing and precipitating factors known to be associated with the development of delirium may be influenced by the use of both pharmacological and nonpharmacological measures.^[7,8]

Although the pathophysiological mechanisms causing delirium are not completely understood, several risk factors have been associated with delirium onset.^[9] Prediction models for ICU delirium, including the recalibrated PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models, have been developed.^[10–12] The E-PRE-DELIRIC model uses only data available at admission, whereas the other 2 models also use data collected during the first 24 hours of the ICU stay.

To determine which delirium prediction model is most appropriate for clinical practice, we need to compare 3 good models.

All 3 models have been validated both internally and externally, resulting in fair to high overall accuracy (area under the receiver operating characteristic [ROC] curve [AUC], 0.63–0.78).^[13–18] However, the Lanzhou model has been externally validated only once. It is therefore appropriate to perform a study comparing all 3 models.

Mechanical ventilation has been identified as one of the prognostic factors for the development of delirium^[9] but is not included as a predictive variable in the PRE-DELIRIC model. Because not all ICU patients receive mechanical ventilation during the first 24 hours after admission, it is currently unknown whether the prognostic accuracy of the PRE-DELIRIC model varies between patients receiving and patients not receiving mechanical ventilation. Although this model was developed for ICU patients, it might be able to be used outside the ICU, especially in departments with critically ill patients who are not receiving mechanical ventilation, such as intermediate care, acute care, cardiac care, or step-down units. Whether such use is appropriate is currently unknown.

The primary objective of this study was to compare the prognostic accuracies of the PRE-DELIRIC, E-PRE-DELIRIC, and Lanzhou models in ICU patients. The secondary aim was to investigate the difference in prognostic accuracy of the PRE-DELIRIC model between patients receiving and patients not receiving mechanical ventilation.

1 2 3 4 5

Next Section

Abstract and Introduction
Methods
Results
Discussion
Conclusion
References
Sidebar

Table. Characteristics and variables in delirious and nondelirious patients
Characteristic or variable	No. (%) of patients^a			P
Characteristic or variable	Total group N = 1353	Delirious n = 410 (30.3%)	Nondelirious n = 943 (69.7%)	P
Age, median (IQR), y	65.0 (53.0–73.0)	67.0 (58.0–75.0)	64.0 (51.0–72.0)	<.001
Male sex	918 (67.8)	297 (72.4)	621 (65.9)	.02
APACHE II score, median (IQR)	18.0 (14.0–25.0)	23.0 (17.0–27.0)	16.0 (13.0–22.0)	<.001
Medical history
History of hypertension	415 (30.7)	147 (35.9)	268 (28.4)	.007
History of alcohol abuse	125 (9.2)	47 (11.5)	78 (8.3)	.07
History of cognitive impairment	12 (0.9)	5 (1.2)	7 (0.7)	.36
History of delirium	8 (0.6)	3 (0.7)	5 (0.5)	.71
History of dementia	3 (0.2)	1 (0.2)	2 (0.2)	>.99
Patient classification
Medical	585 (43.2)	215 (52.4)	370 (39.2)	<.001
Surgical	502 (37.1)	108 (26.3)	394 (41.8)	<.001
Neurologic/neurosurgical	198 (14.6)	69 (16.8)	129 (13.7)	.31
Trauma	68 (5.0)	18 (4.4)	50 (5.3)	.59
Emergency admission	1000 (73.9)	353 (86.1)	647 (68.6)	<.001
Emergency surgery	258 (19.1)	90 (22.0)	168 (17.8)	.08
At admission
Respiratory failure	783 (57.9)	312 (76.1)	471 (49.9)	<.001
Use of corticosteroids	220 (16.3)	69 (16.8)	151 (16.0)	.75
Mean arterial pressure, median (IQR), mm Hg	77 (67–88)	78 (69–89)	76 (66–88)	.06
Serum urea nitrogen, median (IQR), mmol/L	3.1 (2.2–4.6)	3.4 (2.4–5.3)	2.9 (2.1–4.4)	<.001
Medication during the first 24 hours
Use of sedatives	1107 (81.8)	367 (89.5)	740 (78.5)	<.001
Cumulative morphine dose, median (IQR), mg	20.0 (5.0–122.8)	95.8 (9.9–200.6)	15.0 (2.5–92.5)	<.001
Use of dexmedetomidine	4 (0.3)	1 (0.2)	3 (0.3)	>.99
Within the first 24 hours
Mechanical ventilation	1114 (82.3)	366 (89.3)	748 (79.3)	<.001
Metabolic acidosis	763 (56.4)	270 (65.9)	493 (52.3)	<.001
Infection	554 (40.9)	195 (47.6)	359 (38.1)	.001
Coma	534 (39.5)	236 (57.6)	298 (31.6)	<.001
Highest urea concentration, median (IQR), mmol/L	7.4 (5.2–11.0)	8.1 (5.9–12.3)	7.1 (5.1–10.4)	<.001

Abbreviation: APACHE, Acute Physiology and Chronic Health Evaluation.
^aUnless otherwise indicated.

Supplemental Table 1. Variables included in the 3 intensive care unit (ICU) delirium prediction models
PRE-DELIRIC	E-PRE-DELIRIC	Lanzhou
Age	Age	Age
APACHE II^a		APACHE II
Coma^b		Coma
Patient classification (medical, surgical, trauma, neurologic)	Patient classification (medical, surgical, trauma, neurologic)	Multiple trauma
Emergency admission^c		Emergency surgery
	History of cognitive impairment^d	History of dementia
	History of alcohol abuse	History of delirium
		History of hypertension
	Presence of respiratory failure^e	Mechanical ventilation
Metabolic acidosis^f		Metabolic acidosis
	Use of corticosteroids^g
Use of sedatives^h		Use of dexmedetomidine
Morphine doseⁱ
Urea concentration^j	Serum urea nitrogen at ICU admission^k
	Mean arterial pressure at admission
Presence of infection^l

^aAcute Physiology and Chronic Health Evaluation II score, calculated after 24 hours.
^bRichmond Agitation-Sedation Scale (RASS) score of –4 or –5 for more than 8 hours, or the use of more than 2500 mg of propofol or 100 mg of midazolam in the first 24 hours of admission, or a score of 3 on the EMV score (eye opening, best motor response, best vocal response) without sedation.
^cUnplanned admission to ICU.
^dKnown medical history of dementia, mild cognitive impairment, or delirium.
^eNonelective mechanical ventilation or noninvasive ventilation is necessary or expected within 24 hours.
^fpH < 7.35 with bicarbonate < 24 mmol/L.
^gOral or intravenous, used until time of admission to ICU.
^hDichotomized any use of propofol, midazolam, lorazepam, or combination.
ⁱCumulative dose: 0 or 0.01–7.1 mg, 7.2–18.6 mg, ≥18.7 mg.
^jHighest value in blood.
^kUrea/2.14, in mmol/L.
^lProven or strong suspicion of infection for which antibiotics were started.

Supplemental Table 2. Comparison of scores in the models between delirious and nondelirious patients
Model and score	No. (%) of patients^a		P
Model and score	Delirious (n = 410)	Nondelirious (n = 943)	P
PRE-DELIRIC score in probabilities, median (IQR)	0.42 (0.29–0.56)	0.25 (0.18–0.40)	<.001
PRE-DELIRIC score, number of patients per risk group
Low (0.00–0.20)	31 (7.6)	287 (30.4)
Moderate (0.20–0.40)	152 (37.1)	411 (43.9)
High (0.40–0.60)	142 (34.6)	179 (19.0)
Very high (>0.60)	85 (20.7)	66 (7.0)
E-PRE-DELIRIC score inprobabilities, median (IQR)	0.32 (0.20–0.40)	0.19 (0.09–0.31)	<.001
E-PRE-DELIRIC score, number of patients per risk group
Low (0.00–0.20)	97 (23.7)	478 (50.7)
Moderate (0.20–0.40)	207 (50.5)	358 (38.0)
High (0.40–0.60)	97 (23.7)	100 (10.6)
Very high (>0.60)	9 (2.2)	7 (0.7)
Lanzhou score in probabilities, median (IQR)	0.45 (0.38–0.49)	0.40 (0.32–0.45)	<.001
Lanzhou score, number of patients per risk group
Low (0.00–0.20)	5 (1.2)	76 (8.1)
Moderate (0.20–0.40)	109 (26.6)	370 (39.2)
High (0.40–0.60)	287 (70.0)	491 (52.1)
Very high (>0.60)	9 (2.2)	6 (0.6)

^aUnless otherwise indicated.

Supplemental Table 3. Area under the receiver operating characteristic curve cut points for the models
Model and cut points	Sensitivity, %	Specificity, %	Positive likelihood ratio	Negative likelihood ratio
PRE-DELIRIC
Low (0.1)	>99	5	1.0	0.0
Moderate (0.3)	72	59	1.8	0.5
High (0.5)	34	87	2.6	0.8
Very high (0.7)	8	97	2.9	0.9
E-PRE-DELIRIC
Low (0.1)	87	34	1.3	0.4
Moderate (0.3)	55	74	2.1	0.6
High (0.5)	6	97	2.1	1.0
Very high (0.7)	0	>99	—	1.0
Lanzhou
Low (0.1)	100	0	1.0	0.0
Moderate (0.3)	90	20	1.1	0.5
High (0.5)	20	89	1.8	0.9
Very high (0.7)	<1	>99	2.3	1.0

^aUnless otherwise indicated.

processing....

Comparison of PrognosticAccuracy of 3 Delirium Prediction Models

Abstract and Introduction

Abstract

Introduction

Tables

Tables

Tables

Tables

References

Authors and Disclosures

Authors and Disclosures

Sidebar

Sidebar

See Also

Email This

Feedback