Introduction

Coronavirus disease 2019 (COVID-19) occurred with different levels of severity, and different criteria for hospitalizations were suggested, considering risk factors for poor outcomes, such as advanced age, comorbidities, lymphopenia, and high C-reactive protein (CRP) levels (1). The most severe complication of COVID-19 is viral pneumonia, which is often manifested by diffuse bilateral lung involvement (1,2).

Various prognostic scales were developed to evaluate the indication for hospital or ICU admission and the severity of the disease in community-acquired pneumonia, and the suitability of these scales in patients with primary viral pneumonia was investigated (3). For COVID-19 pneumonia, non-specific predictive scales were evaluated (2). However, specific predictive risk estimation scores for COVID-19 pneumonia were needed to facilitate clinicians’ decisions regarding hospitalization and admission location. Accordingly, a study by Rodriguez-Nava et al. evaluated the performance of the CURB-65 score, the quick COVID-19 severity index (qCSI) score, and the Brescia-COVID Respiratory Severity Scale (BCRSS) score in predicting in-hospital mortality and ICU admission in patients with COVID-19 illness (4). This study sought to provide a cohort validation of the previous study for further evidence.

Severity scores are valuable tools to predict in-hospital outcomes and guide clinicians’ treatment decisions. For example, the CURB-65 score (confusion, BUN >19 mg/dL, respiratory rate ≥30, low blood pressure, and age >65 years) used in the emergency department (ED) was developed as a clinical prediction rule in patients with pneumonia. The CURB-65 score can be easily measured during the first evaluation of patients in ED (5–7). Moreover, two prognostic risk scores developed specifically for COVID-19 and easy to use in the ED are of interest: the qCSI and the BCRSS. The qCSI score predicts critical conditions defined by high-flow oxygen demand, non-invasive or invasive ventilation, or death in the first 24 hours (8). On the other hand, the BCRSS includes respiratory rate, oxygen saturation, and respiratory examination characteristics of the patient and offers treatment recommendations (9).

Our study aimed to evaluate the performance of the CURB-65, the qCSI, and the BCRSS scores in predicting ICU hospitalization and in-hospital mortality in emergently hospitalized patients with COVID-19 pneumonia.

Materials and Methods

We conducted a retrospective study in Kayseri City Hospital between March 2022 and April 2022. Kayseri City Hospital’s Ethics Committee approved this study on Jun 2, 2022, with the approval code 647. We waived informed consent because of the retrospective nature of the study.

Study Population

We retrospectively reviewed the emergently hospitalized patients with COVID-19 pneumonia consecutively. A confirmed case of COVID-19 was defined as a positive reverse transcriptase-polymerase chain reaction (RT-PCR) analysis of a nasopharyngeal swab sample. We included only laboratory-confirmed cases that had pulmonary involvement. The primary endpoint of our study was in-hospital mortality and ICU admission.

Severity Scores

CURB-65 Score: The CURB-65 score (confusion, BUN >19 mg/dL, respiration speed >30/min, low blood pressure (systolic BP <90 mm Hg or diastolic BP <60 mm Hg), age >65 years), which is used to determine the need for hospitalization in adults diagnosed with community-acquired pneumonia, has a low level of evidence. The CURB-65 score varies from 0 to 5. A score between 0-1 indicates a low mortality risk, while 2 and above indicate higher mortality (10).

Quick COVID-19 Severity Index (qCSI): The qCSI is a 12-point scale that uses only three variables available at the bedside: 1) nasal cannula flow rate, 2) respiratory rate, and 3) the minimum measured pulse oximetry value. According to scores, there are four risk strata: 0-3 low, 4-6 low-intermediate, 7-9 high-intermediate, and ≥10 high risks (8).

Brescia-COVID Respiratory Severity Scale (BCRSS): The BCRSS used during the initial assessment includes the following risk factors: 1) the presence of wheezing or when the patient is unable to complete a full sentence at rest or with minimal effort, 2) the respiratory rate is greater than 22, 3) the oxygen saturation is below 90%, and 4) the presence of bilateral or diffuse infiltrates on lung imaging (9). Patients are classified into five risk tiers based on these risk factors. 

Sample Size

Based on the previously mentioned research (4), we calculated the required sample size to compare the areas under the two ROC (receiver operating characteristic) curves (AUC) using the MedCalc 20.0 program (MedCalc Software Ltd., Ostend, Belgium). Assuming that the correlation is 0.4 in both the positive and negative cases, and the sample size ratio in the negative/positive groups is 2 (i.e., the mortality rate is 33%), we calculated a total sample size of 243 at α= 0.20 and 95% power.

Data Collection

We conducted a retrospective study between Mar 1, 2022, and Apr 1, 2022, and the ED doctors recorded patients’ medical data on the data collection form. The patients’ fever, pulse, blood pressure, respiratory rate, and oxygen saturation values were recorded in the medical data by the emergency nurses. The ED physician recorded the patient’s state of consciousness, the amount of oxygen taken by nasal or mask, and detailed respiratory physical examination findings in the patient’s medical data. The principal abstractor recorded the BUN value from the laboratory values of the patients in the data collection form. The radiology physician reported the lung imaging findings of the patients (all patients had thorax tomography) according to the COVID-19 Reporting and Data System (CO-RADS) classification system (11). Study patients had all CO-RADS 4 or 5. Two blinded infectious disease physicians calculated the scores using data collection forms and medical data in the study.

Statistical Analysis

Statistical analyzes were performed using the Statistical Package for the Social Sciences (SPSS) 26.0 (IBM Corp., Armonk, NY, USA)  and MedCalc 20.0 program (MedCalc Software Ltd., Ostend, Belgium). We showed the categorical data as frequency and percentage, the continuous data as the mean ± standard deviation for the normally distributed data, and the median 25%-75% interquartile ranges (IQRs) for non-normally distributed. We used the Chi-square test for categorical variables. We used the Mann-Whitney U test for two-group comparisons of non-normally distributed continuous variables and independent sample t-tests to compare normally-distributed continuous variables. We performed ROC analyses of CURB-65, qCSI, and BCRSS scores and compared the ROC curves of these three scores. For descriptive statistics, AUC value, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), and negative likelihood ratio (LR–) are shown. Statistical significance was set as p<0.05.

Results

During the study period, 1162 patients visited the pandemic ED; 378 were hospitalized with a preliminary diagnosis of COVID-19. Ninety-two patients were excluded from the study because they tested negative in PCR, and twenty-eight patients were excluded because of missing data. A total of 258 patients consecutively hospitalized were included in the study. Chest computed tomography (CT) was performed in all patients as pulmonary imaging. 

Table 1. Demographic and clinical characteristics of emergently hospitalized patients with COVID-19 pneumonia.

The median age of the patients was 73 years IQRs (64- 82) (min: 24 max: 110), and 63.6% (n=164) were male. 29.5% (n=76) of the patients were admitted to the ICU, and 15.9% (n=41) died. The most common comorbidities were hypertension (39.1%), chronic obstructive pulmonary disease (COPD) (36.7%), and diabetes mellitus (30.2%). We showed patients’ characteristics in Table 1.

There was no significant difference between gender and comorbidities regarding ICU admission and in-hospital mortality, except hypertension in in-hospital mortality. However, the mean age of patients was higher in patients admitted to ICU and dead (ICU admission: 74.5 vs. 70.4; in-hospital mortality: 77 vs. 70.6; p=0.029 and p=0.007, respectively). The median length of stay (LOS) in the hospital was higher in patients admitted to ICU and dead (p≤0.001 and p=0.015, respectively). There was a significant difference between the CURB-65, the qCSI, and BCRSS scores regarding ICU admission and in-hospital mortality (p<0.005) (Table 1).

The Performance of the Scores

Table 2. CURB-65, qCSI, and BCRSS scores’ power to predict ICU admission.

Table 3. CURB-65, qCSI, and BCRSS scores’ power to predict death.

Figure 2. ROC curves for death for the CURB-65, qCSI, and BCRSS scores.

The sensitivity, specificity, PPV, NPV, +LR, and -LR values of the three scores in terms of predicting ICU admission and in-hospital death at different threshold values are presented in Tables 3 and 4.

Overall, the predictive effect of the CURB-65 score was better than the qCSI and the BCRSS scores regarding ICU admission and in-hospital mortality. The CURB-65 and the qCSI scores predicted ICU admission at a moderate level (p≤0.001; AUC values were 0.743 and 0.723, respectively). However, the predictive effect of the BCRSS score for ICU admission was lower  (p≤0.001; AUC value was 0.667). The CURB-65 score showed no significant difference from the qCSI and the BCRSS scores regarding the predictive effect for ICU admission (p=0.6268 and p=0.0802, respectively) (Figure 1). When the CURB-65 score was higher than 2, it had a sensitivity of 48.6%, a specificity of 89%, a PPV of 64.9%, a NPV of 80.6%, a positive likelihood ratio of 4.4, and a negative likelihood ratio of 0.58 for predicting ICU admission. Besides, when the qCSI score was higher than 5, it had a sensitivity of 84.2%, a specificity of 55.4%, PPV of 44.1%, NPV of 89.4%, LR+ of 1.89, and LR- of 0.28 for predicting ICU admission. (Table 2, Figure 1).

The CURB-65 predicted in-hospital mortality at a moderate level (p≤0.001; AUC value was 0.762). On the other hand, the predictive effect of the qCSI and the BCRSS scores for in-hospital mortality were low (p≤0.001 and p=0.012, respectively; AUC values were 0.655 and 0.612, respectively). In addition, the CURB-65 score significantly differed from the qCSI and BCRSS scores regarding the predictive effect for in-hospital mortality (p=0.0061 and p=0.0019, respectively) (Figure 2). The CURB-65 score higher than 2 had a sensitivity of 58.5%, a specificity of 84.7%, PPV of 42.1%, NPV of 91.5%, LR+ of 3.85, and LR- of 0.49 for predicting in-hospital mortality (Table 3, Figure 2).

Discussion

Our study evaluated the performance of the CURB-65, the qCSI, and the BCRSS scores in predicting ICU admission and in-hospital mortality in patients with COVID-19 pneumonia. The CURB-65 had the highest AUC for predicting ICU admission and in-hospital mortality. Also, the qCSI score had a good AUC for ICU admission.

Mortality rates in COVID-19 have been reported at different rates, such as 4.2% and 20.9%, in various studies (1, 12). However, in our study, in-hospital mortality was high (15.9%) because the patient’s median age was 73 years, and they had pneumonia.

The CURB-65 score predicts the 30-day fatality in community-acquired pneumonia to aid in identifying inpatient or outpatient treatment, and it is also used in COVID-19 (13). Various studies have demonstrated that the CURB-65 score is a good predictor of in-hospital mortality in COVID-19 (3, 14). The qCSI (including respiratory rate, pulse oximetry, and oxygen flow rate), a simple scoring system, predicts a 24-hr risk of critical respiratory illness in patients admitted from ED with COVID-19 (8). In a retrospective study of 313 patients, while the CURB-65 and qCSI scoring systems performed well in predicting in-hospital mortality, the qCSI and the BCRSS scores performed well in predicting ICU admission (4). In our study, the CURB-65 score predicted ICU admission and in-hospital mortality better than the qCSI and the BCRSS scores. Nevertheless, the qCSI score predicted ICU admission better than the BCRSS score.

The BCRSS score, developed by the Italian Society of Infectious and Tropical Diseases, assesses the respiratory severity of COVID-19 pneumonia, showing the patient’s need for oxygen and mechanical ventilation (15). It was demonstrated to be good at predicting severe illness and mortality (16). However, in our study, the predictive effect of the BCRSS score for ICU admission and in-hospital mortality was low.

Our study had some limitations. Firstly, we conducted a single-center and retrospective study. Secondly, we examined only three scoring systems for estimating the severity of COVID-19 disease. In addition, we evaluated all measures only once at admission, so we could not assess changes in these scores.

In conclusion, the qCSI score performed well in predicting ICU admission, and the CURB-65 score predicted ICU hospitalization and in-hospital mortality. In contrast, the BCRSS score is poorly predictive of ICU admission and in-hospital mortality. Nevertheless, we recommend that future studies evaluate the value and utility of COVID-19 risk classification models.