Comparison of Paediatric Index of Mortality 3, Paediatric Risk of Mortality III, Paediatric Logistic Organ Dysfunction-2 for Assessing Patient Mortality: A Prospective Observational Study

1. Senior Resident, Department of Paediatrics, SCB Medical College and Hospital, Cuttack, Odisha, India. 2. Professor, Department of Paediatrics, SCB Medical College and Hospital, Cuttack, Odisha, India.

Abstract

Introduction: Numerous scoring systems have been proposed in an effort to increase the prognostic accuracy and predicting outcome. In order to measure the risk of mortality, scores are employed that establish a numerical scale and in this way, they compare estimated mortality in % with the observed mortality. Known as prognostic scores, these can be used to evaluate the quality of medical care and to optimise the employment of resources, aiming at improving the cost-benefit relationship. Since, they compare mortality adjusted by disease severity these scores can also be used for comparisons between clinical trials and for planning technological resources.

Aim: To compare the performance of the Paediatric Risk of Mortality III (PRISM III), the Paediatric Index of Mortality 3 (PIM 3) and Paediatric Logistic Organ Dysfunction-2 (PELOD-2) scores in a Paediatric Intensive Care Unit (PICU) in a tertiary care hospital.

Materials and Methods: The present study was prospective observational study which included children from one month to 14 years of age admitted to PICU, and who remained in PICU after 24 hours. Within the first hour of admission PIM 3 was assessed. Further at 24 hours of admission, PRISM III and PELOD-2 score were assessed. Performance of different scores were evaluated. Calibration by HosmerLemeshow goodness-of-fit test {χ²(p)} Discrimination was assessed by the ROC curve. Standardised Mortality Rate (SMR) was calculated to predict the mortality.

Results: Total 281 children were enrolled in the study, out of which 62 patients died. Neurological illness was the most common cause of death (12, 19.35%) followed by respiratory and haemato-onco cases (10, 16.13%) each. The Area Under the ROC Curve-Receiver Operating Characteristics (AUC-ROC) of PELOD-2, PIM 3 and PRISM III were 0.862, 0.847 and 0.838, respectively. Among the three scores PELOD-2 had poor calibration for the study population (χ²=18.837, p=0.016, d=8). PIM 3 was a better predictor of mortality (with SMR of 1.33) when compared with PRISM III and PELOD-2 (which had SMR of 1.57 and 1.83, respectively).

Conclusion: All the three scores had good discrimination, however PELOD-2 had poor calibration for the given study population, with respect to better predictor of mortality all the scores underestimate the mortality. Among these, the better predictor mortality was PIM 3. Since, PIM 3 also had good calibration for the study population and is associated with less variables to monitor there is ease of estimation and hence it is more suitable to score and to assess mortality.

Morbidity, Paediatric critical care, Prognosis

In intensive care, a rational and objective way to define and quantify severity of illness is through the development of probability models predicting mortality risks. Such predictive models or scoring system have been developed for all age groups including paediatrics (1),(2). Risk-adjustment tools that predict death in PICUs have become established only in the past 30 years (3). Patient’s mortality is not only affected by Intensive Care Unit (ICU) performance but also depends on many other factors such as demographic and clinical characteristic of population, infrastructure and non medical factors (management and organisation), case mix and admission practice (4). The capacity to estimate patient’s risk of mortality is extremely important because such estimate would be useful in achieving many different goals such as assessing patient’s prognosis, ICU performance, ICU resource utilisation, evaluating therapies, and also controlling and matching severity of illness in clinical studies (5).

The principal scores that have been developed for the paediatric population are the PRISM (6), PIM, PELOD, pSOFA (paediatric Sequential Organ Failure Assessment) and many more. Their most recent versions being PRISM III, PIM 3, PELOD-2 (7),(8),(9). The advantages of the scoring system are that they can be used to evaluate the quality of care provided in the medical facility, resource management and aiming to improve benefits for the patient with reducing the financial burden to the management. These scores have been validated for their accuracy and reproducibility in various PICU setting for individual disease and individual scores (10),(11) and practically used to compare clinical trials. To date, the studies that have been performed independently, have not used heterogenic groups of patients from PICUs, but have investigated certain specific disease categories, new versions of the methods or homogenous groups of high mortality patients. In this independent study, the objective was to compare the performance of the PRISM III, PIM 3 and PELOD-2 at a general PICU.

These scores involve different variables which may overlap sometimes and few have temporal association with arrival to PICU (such as PIM is done at arrival whereas PRISM and PELOD are done at 24 hours of arrival). Moreover, these scores were developed in the western society and have been validated extensively in their settings. The scenario of their validation may be different from Indian circumstances for both clinician and hospital management which once validated will help the clinician triage his resource for optimum outcome and the policy makers to allocate resources efficiently. However, very few Indian studies are available for the validation of these scores. Tyagi P et al., showed PRISM III and PIM 3 had good calibration as well as good discrimination but had not included PELOD-2 in their study (12). The study done in southern Indian state of Kerala by Ali NK et al., had PRISM III predicting the outcome in the PICU with good discrimination and calibration as well but was having a smaller sample size and the study was done only with PRISM III (13). Individual institution practices can influence the outcome, and this is why each centre needs to validate the scores. Authors, therefore planned this study at a tertiary care PICU in Eastern India.

Material and Methods

This was a prospective observational study conducted in a tertiary care hospital. The study was done between January 2019 to December 2019. The Ethical Committee clearance was obtained vide letter number 896.14.10.2019.

Sample size calculation: Sample size was calculated based on the mortality rate of PICU as reported by Roy SM et al., in a tertiary care hospital from Eastern India (14). The mortality was 24%, using Fischer formula and assuming 95% confidence interval and alpha error of 5%:

Sample size (N)= 1.96×1.96 {0.24}×{0.76 / (0.05)² =280

Inclusion criteria: Children from one month of age to 14 years of age group admitted to the PICU were enrolled in the study. (based on consensus guideline for PICU admission by Indian Society of Critical Care Medicine) (15).

Exclusion criteria: Patients who remained in ICU for <24 hours, patients with multiple congenital anomalies and parents not consenting for the study were excluded from it.

Total 281 children were included in the study. After admission to PICU, detailed history was collected and data collected regarding age, sex, weight, duration of illness with clinical diagnosis. Relevant investigations were done as per indication and treatment of patients. Within 1^st hour of admission PIM 3 was assessed, and calculated using 10 physiological variables (16), at 24 hours PRISM III score was assessed and calculated using 17 physiological variables (17) and PELOD-2 scores was assessed and calculated using 5 organ dysfunctions and 10 variables (9).

Statistical Analysis

Categorical variables were expressed as number of patients and percentage of patients and compared across the groups using Pearson’s Chi-square test for independence of attributes/fisher’s-exact test as appropriate. Continuous variables were expressed as mean, median and standard deviation and compared across the groups using Mann-Whitney U test. Performance of the scoring systems was evaluated by calibration and discrimination. SMR was calculated for the given population. Discrimination was assessed by the Area under Receiver Operating Characteristics (ROC) curve which indicates the accuracy and the efficacy of various scores to discriminate between the survivors and non survivors, this measurement was used to predict death.

Calibration of the scoring system was assessed by Hosmer-Lemeshow goodness-of-fit test {χ²(p)}. Hosmer-Lemeshow goodness-of-fit tests is used for evaluating the calibration of the various scoring systems, this test of significance suggest the score that had the least statistically significant discrepancy between predicted and observed mortality. The p-value >0.05 is least statistically significant and hence better calibrated. The statistical software Statistical Package for the Social Sciences (SPSS) 20.0 was used for analysis.

Results

Out of total 321 children admitted to the PICU, 40 children could not be followed and were lost or excluded from the study. So, 281 children were enrolled in the study, out of which 62 children died, thus the crude mortality rate of 22.1% (Table/Fig 1). In the study populations, most of the children were critical and belonged to age group of 1 month to 1 year (135, 48.04%). The mortality highest in this age group (31,50%). This result was statistically not significant. In the study more male children got admitted to the PICU (150, 53.38%) than female (131, 46.62%), however the difference was not statistically significant. Among those who died, majority had a hospital stay for more than seven days (n=32, 51.61%) (Table/Fig 2).

(Table/Fig 3) depicts the overall performance of individual scoring system. PIM 3 had the nearest estimate of mean mortality 19.86% to the observed crude mortality of 22.1% and so it’s SMR was 1.33. However, including, PIM 3 all the scoring system underestimated the risk of mortality.

The Hosmer and Lemeshow goodness of fit test showed a good calibration for PIM 3 (χ²=7.292, p=0.505), PRISM III score (χ²=11.868, p=0.105), but showed poor calibration for PELOD-2 (χ²=18.837, p=0.016) and was not a good fit scoring system for the study population. PELOD-2 showed good discrimination among the survivors and non survivors with AUC=0.862 (CI=0.808-0.915) and rightly so did PELOD-2 outperformed in discriminating survivors and non survivors when compared with PIM III and PRISM III (AUC=0.847 and AUC=0.838, respectively) (Table/Fig 4).

Children with neurological ailments were among the highest to be admitted in the PICU (70, 24.91%), and so most common illness associated with mortality in the study was Neurological Diseases n=12 (19.35%). This was followed by respiratory and haemato-onco cases (each with10 deaths), next was cardiovascular involvement (8, 12.9%) (Table/Fig 5).

Discussion

The principal scores that have been developed for the paediatric population are the PRISM, PIM, PELOD with their most recent versions being PRISM III, PIM 3 and PELOD-2. The present study investigated the relationship between observed mortality and survival with the predicted mortality and survival rates as estimated by the three scores. Calibration compares the expected and observed mortality at various intervals of severity whereas discrimination distinguishes the outcome as either survivor or non survivor.

This study done was in a tertiary care hospital of Eastern part of India, had good discrimination and calibration. PIM 3, PRISM III and PELOD-2 had good discrimination as these had AUC-ROC of 0.847, 0.838 and 0.862, respectively. A good ROC means the value should be >0.80. The study done by Rady HI et al., found a ROC of 0.75, 0.747 and 0.732 for PRISM III, PIM 2 and PELOD-2 respectively which were fair (18). Sankar J et al., in their study in AIIMS, New Delhi, India had shown ROC of 0.75 for PIM 3 and 0.69 for PIM 2 score (19). Jung JH et al., in their study in Seoul, Korea had shown ROC of 0.826 for PIM 3 and 0.0775 for PRISM III, respectively (20). An ROC of 0.87 and 0.85 were calculated by Martha VF et al., for PRISM and PIM respectively in their study (21). Gonçalves JP et al., had ROC of 0.92 for PRISM III and 0.94 for PELOD-2 in their study (22). The SMR obtained was 1.33 according to PIM 3 with the 95% confidence interval being 0.91-1.63 which predicted 86.29% of mortality.

However, PRISM III was able to predict mortality with the SMR of 1.57. SMR close to one signifies better prediction. If significantly more than one suggest that the performance of ICU can be improved and there is underestimating of the mortality. Jung JH et al., had SMR of 1.11 for PIM 3 (20). Similar results were obtained by Sari DSP et al., in their study they had an observed mortality of 40.58% with SMR being 2.25 (23). Raghavendra J et al., also observed in their study that PIM 3 underestimated mortality, with an observed mortality rate of 9.3% with SMR of 2.02 (24). Since, there was good calibration in the scores PIM III and PRISM II which is suggested by Hosmer-lemeshow goodness-of-fit test p>0.05; meaning the observed mortality data is not any different from the expected mortality. Similar results were obtained by Tyagi P et al., in their study conducted in Western India (12). Since the p-value <0.05 for PELOD-2 for the same test of significance it meant the expected mortality data is significantly different from the observed mortality so PELOD-2 had poor calibration for the PICU in the study.

When compared to western countries which usually overestimate the mortality, this is not so in developing nation and in particular to the index PICU in Eastern India which underestimated the mortality. One such Italian study had observed mortality of 4.4% (95% CI, 3.7-5.2), compared to 6.4% (95% CI, 5.5-7.3) expected mortality according to PIM 2 with SMR of 0.7 (95% CI: 0.6-0.8) which overestimated the mortality (25). Another European study done in Netherlands by Gemke RJ and van Vught J had crude observed mortality of 20 (6.6%) in the PICU with expected mortality based on PRISM III (24 hours) was 6.95% (SMR 0.95; 0.67-1.22) which also overestimated the mortality (26).

However, studies done in developing nation underestimated the mortality. This finding was similar to studies done in other part of India be it, Tyagi P et al., in Western India or Sankar J et al., in Northern part of India (12),(19). This is because mortality rate differs in developed country and developing country at various level of severity, population characteristics and standard of care provided. Despite the best possible care underestimation of mortality occurs which suggest that the scores should be standardised accordingly to a developing nation.

Overall, since these scores do have excellent discrimination, they can be used to evaluate the overall performance of the PICU and for individual patient application, since PIM 3 and PRISM III has good calibration as well as excellent discrimination it is suitable.

In a busy PICU, monitoring the patient’s course in ICU is of utmost importance. In such situation, data compilation and interpretation should not only be quick but also accurate. PIM 3 with collection of 10 physical variables has better ease of data compilation as compared to 17 physical and biochemical variables of PRISM III. Since PIM 3 scoring is also done at admission it is not only simple but also quick in computing the results while PRISM III is done 24 hour after admission.

Limitation(s)

It was a single PICU study, multi-unit ICU study are required to address these problems.

Conclusion

The PIM 3 model is best model for mortality prediction and it has good discrimination and calibration for PICU with SMR obtained of 1.33. As SMR is >1, it suggests that the mortality is not just dependent on the admission characteristic of the critical illness, there are other preadmission events which might influence the outcome. PIM 3 in present scenario is underestimating the mortality however it is very close to the mortality occurred. Since, PIM 3 estimation of mortality involves less variable it is more suitable in busy PICU. PELOD-2 was discarded as it had poor calibration.

DOI and Others

DOI: 10.7860/JCDR/2022/51228.15886

Date of Submission: Jul 04, 2021
Date of Peer Review: Oct 14, 2021
Date of Acceptance: Nov 19, 2021
Date of Publishing: Jan 01, 2022

AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. NA

PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Jul 09, 2021
• Manual Googling: Nov 18, 2021
• iThenticate Software: Dec 08, 2021 (16%)

ETYMOLOGY: Author Origin