The Official Journal of the Turkish Society Of Clinical Microbiology and Infectious Diseases (KLİMİK)

Original Article

All-Cause Excess Mortality in 2020: The Example of Bursa City in Turkey

Kayıhan Pala
×Affiliations
  • Public Health Department, Bursa Uludağ University School of Medicine, Bursa, Turkey
,
Neşe Yürekli
×Affiliations
  • Public Health Department, Bursa Uludağ University School of Medicine, Bursa, Turkey
,
Nil Kader Çağaç
×Affiliations
  • Public Health Department, Bursa Uludağ University School of Medicine, Bursa, Turkey
,
Alpaslan Türkkan
×Affiliations
  • Public Health Department, Bursa Uludağ University School of Medicine, Bursa, Turkey

Abstract

Background: This study aimed to estimate excess all-cause mortality rates in Bursa Province in 2020.

Methods: In this study, a retrospective descriptive analysis of the mortality rates in Bursa, Turkey’s fourth biggest city, between 2015-2020 was conducted. The data were taken from Bursa Metropolitan Municipality death records. Daily mortality data were classified as age, gender, the date and cause of death (communicable or non-communicable disease). An Excel mortality calculator was used to analyze the data and calculate the excess mortality. Excess mortality was calculated with a 5-year death average. The excess mortality-expected death ratio (P-score) was calculated as a percentage difference between the average number of deaths in 2015-2019 and the number of deaths in the same period (week or month) of 2020. The Chi-square test was used for statistical analysis.  

Results: In 2020 in Bursa, crude excess mortality was calculated as 5390 (95% CI: 4525-6256) compared to the previous five years’ average, and the P-score was 35%. Excess mortality decreased in the 0-14 age group in both genders and increased mainly in the 65+ age group. In 2020, 85.3% of the excess mortality was due to communicable diseases, and 4596 (95% CI: 4562-4631) people reported to die due to communicable diseases. Thus, deaths because of communicable diseases increased approximately 76 times in 2020 compared to the previous five years’ average.

Conclusion: Compared to the previous five years’ average, the annual number of deaths in Bursa increased by approximately one-third in 2020, and most of the excess deaths were due to communicable diseases. For the use of revealing the actual burden of the COVID-19 pandemic on all-cause mortality, it is crucial to assess the extreme all-cause mortality. 

Keywords: COVID-19, excess mortality, pandemic, Bursa, Turkey

Highlights

  • Estimating excess mortality is a non-specific measure of pandemics’ severity or effects and public health emergencies.
  • Five thousand three hundred ninety excess deaths occurred in Bursa in 2020 compared to the previous five years (2015-2019).
  • The number of weekly deaths in Bursa peaked at the 48th week in 2020.
  • Most of the excess deaths (85.3%) were because of communicable diseases.

Introduction

Estimations about confirmed COVID-19 deaths may be limited to the testing capacity and frequency, along with correct and complete reporting of the cause of deaths (1). The number of confirmed and probable COVID-19 deaths may not include people who did not obtain a test/received false-negative results/got infected after testing negative, or people who died outside of the healthcare system (2). An overwhelmed healthcare system or the fear of contracting the disease may also alter health-seeking behaviors and increase non-COVID-19 related deaths (3).

Estimating excess mortality, especially where asymptomatic cases are present and diagnostic and reporting systems are inadequate, is a non-specific measure of pandemics’ severity or effects and public health emergencies (4). So, reporting the excess number of deaths may provide a more accurate estimate of the pandemic’s results (2).

The term “excess mortality” was first used by Collins in 1932 while investigating the increase in influenza and pneumonia-related deaths during the 1918 Influenza Pandemic (5). This concept is defined as the difference between the number of deaths observed in a certain period and the number of expected deaths in the same period (6). Today, it is recommended as one of the most appropriate indicators to measure the burden of the COVID-19 pandemic on all-cause death (7).

The calculations were made for comparing the deaths in the last five years in the countries severely affected by the pandemic. For example, 47,490 (8) deaths were detected during the first wave in Italy excess. In the USA, 538,615 (6) excess deaths were seen between February 1, 2020, March 13, 2021, and 101,852 (9) excess deaths were reported in the UK between March 21, 2020, and February 26, 2021. Thus, the numbers reported by the countries as confirmed COVID-19 deaths are pretty low compared to the calculated excess number of deaths. 

The first confirmed case in Turkey was reported on March 11, 2020, the same day the World Health Organization (WHO) characterized COVID-19 as a pandemic. After that, the number of cases reached 2208 652 until the end of the year. After March 17, when the first confirmed COVID-19 death was declared, 20,881 deaths were reported by the end of the year (10). Because only confirmed cases are reported as COVID-19 deaths in Turkey, deaths due to COVID-19 are thought to be underreported than they are (11).

In addition to the low sensitivity and selectivity of the RT-PCR test used for diagnosing COVID-19 (12), the Ministry of Health allowed testing if there was a history of travel abroad, close contact with a case, or at least one of these symptoms: cough, fever, shortness of breath. Many infected people could have been missed because of the strict testing guidelines and the absence of screening in risk groups. We should also note that the Turkish Ministry of Health reported only the confirmed COVID-19 deaths; the distribution of deaths by age, gender and place was not declared. Therefore, it is not known which groups are affected by the deaths related to the pandemic. Because there might be possible/suspicious cases-deaths that were diagnosed clinically and epidemiologically with COVID-19 but not confirmed by testing, the true impact of the pandemic could not be evaluated (13).

This study aimed to estimate the burden of the COVID-19 pandemic in Bursa in 2020 concerning mortality by calculating the number of excess deaths.

Methods

This descriptive retrospective study included the analysis of daily death numbers between 2015-2020 in Bursa, Turkey’s fourth-largest city. Bursa, having 17 districts, is located in the Marmara region in the west of Turkey, and its population was 3,101,833 in 2020 (14).

The data set of Bursa Metropolitan Municipality documenting daily deaths were used to analyze the mortality data. Daily death data were classified according to age, gender, the date of death. Unfortunately, the cause of death was available only as communicable and non-communicable disease in the data set. Although we requested more detailed information about the causes of death from the Provincial Health Directorate and the Turkish Statistical Institute, we could not obtain this information. 

We used an excel mortality calculator to analyze data, calculate excess deaths, and generate figures (15). This tool calculated weekly baseline values by linear regression using the Excel FORECAST function or averages of weekly values of deaths. By calculating standard error and confidence interval according to the values of each week in five years (2015-2019), it obtained the threshold value, which was the weekly upper limit for expected deaths under normal conditions. Next, it obtained the number of excess deaths for each week by separately subtracting the weekly excess deaths from both the threshold value and the baseline from the number of deaths corresponding to the same week of 2020. As a result of these subtractions, values in which the weekly excess deaths were negative or incorrect were corrected as 0. The annual excess mortality was found by adding up these corrected weekly excess deaths. Finally, the excess mortality-expected death ratio (P-score) was calculated as a percentage difference between the average number of deaths in 2015-2019 and the number of deaths in the same period (week or month) of 2020. The Chi-square test was used for statistical analysis. 

Results

Table 1. Distribution of deaths recorded in Bursa between 2015-2020 by years.

The total number of deaths in Bursa between 2015-2020 was 97,674. While the annual average number of deaths was 15,381 (95% CI: 14,515-16,246) in 2015-2019, the number of deaths in 2020 was 20,771 (Table 1). There were no gender data for 3105 (4.0%) people and age data for 489 (0.6%) people who died in 2015-2019. In 2020, gender data of 615 (2.9%) people and age data of 115 (0.5%) people were not available. 

In 2015-2020, the highest number of daily deaths occurred on the 338th day of 2020 (131 people), and the highest number of weekly deaths happened in the 48th week of 2020 (769 people). 

Figure 1. Weekly deaths in Bursa (Previous five years (2015-2019) and 2020).

Compared to the previous five years, the number of weekly deaths showed an increasing trend from the 41st week to the 47th week in 2020, and after two weeks of being on the rise, a decrease was observed after the 49th week (Figure 1). 

Table 2. Distribution of observed and excess mortality in Bursa by gender and age groups.

In Bursa, crude excess mortality was calculated as 5390 (95% CI: 4525-6256) in 2020 compared to the previous five years’ average. Thus, the excess number of deaths is 4330 when calculated from the threshold value and 5447 when calculated from the baseline (Table 2). 

Figure 2. Weekly 2020 deaths compared with the upper and lower limits of the 2015-2019 year averages (95% CI) in different age groups in Bursa province.

Excess deaths occurred mainly in the group over 65 years of age in both genders (Figure 2). In 2020, compared to the average of the previous five years, 2041 (95% CI: 1677-2425) excess deaths in women and 3355 (95% CI: 2966-3797) excess deaths in men were calculated (Table 2). The mortality rate was higher in males in both periods. While the proportion of men among all deaths was 56.7% in the 2015-2019 period, this rate increased to 58.2% in 2020 (χ² = 7.8, p <0.01).

The number of deaths observed in women in 2020 was lower in the 0-14 age group compared to the average of the previous five years (χ² =18.9, p <0.01), while there was no statistically significant difference in the 15-64 and 65+ age groups (p> 0.05). 

The number of deaths observed in men in 2020 was lower in the 0-14 age group (χ² =48.7, p <0.01) compared to the average of the previous five years, however, higher in the 15-64 (χ² =19.2, p <0.01) and 65+ age groups (χ² =41.9, p <0.01). 

Table 3. Distribution of observed and excess mortality according to causes of death and gender in Bursa.

In 2020, the number of excess deaths due to communicable diseases was determined as 4596 (95% CI: 4562-4631), and due to non-communicable diseases were 794 (95% CI: -73-1661) (Table 3). Thus, 85.3% of excess mortality in 2020 was related to communicable diseases. While the mortality rate from communicable diseases among total deaths in the previous five years was 0.4%, this rate increased to 22.4% in 2020 (χ² = 3775.6, p <0.01). While the yearly average of communicable disease-related deaths was 61 in the previous five years, in 2020, it was 4657. Thus, the number increased approximately 76 times in 2020 compared to the average of 2015-2019. 

Figure 3. Weekly communicable diseases-related deaths in Bursa ( Previous five years (2015-2019) and 2020).

In Bursa, weekly deaths from communicable diseases increased from the 13th week of 2020 compared to the previous five years, and after reaching its peak at the 48th week, a decreasing trend was observed towards the end of the year (Figure 3). 

Discussion 

In 2020, 5390 excess deaths occurred in Bursa compared to the previous five years, and the crude death rate, which was 49.6 per ten thousand in 2015-2019, increased to 67.0 per ten thousand in 2020 (Table 2). In Turkey, the excess number of deaths was not declared officially; however, it is known that, when the COVID-19 pandemic started, excess deaths occurred in various countries and cities compared to previous years (1, 3, 6, 8, 16-18).

In Istanbul, excess mortality was identified in two separate studies conducted at different times in 2020 (19, 20). However, the only information regarding Bursa was obtained when the Mayor of Bursa announced the number of people who died because of COVID-19 at the City Council meeting on December 17, 2020 (21). With this announcement, the death rate related to COVID-19 in Bursa was calculated as 1133 per million. This rate was much higher than Turkey’s COVID-19 death rate at the end of 2020 (248 per million) (22). Besides, the Turkish Minister of Health reported that Istanbul became the “Wuhan of Turkey” (23) with the highest number of cases and deaths but did not share any information about the other provinces. The COVID-19 mortality rate in Bursa was very high compared to Turkey in general, and it was unexpected. Therefore, the debate that the Ministry of Health was underreporting COVID-19 deaths in Turkey was raised (11, 24). 

According to this study, the highest number of daily and weekly deaths in Bursa between 2015-2020 was reported in 2020. Similarly, in Sweden, 18% excess deaths occurred in 2020 compared to 2010-2019 (25). With the highest number of COVID-19 deaths in 2020, Spain had the highest excess mortality rate among European countries with 29.1% (25). Compared to 2016-2019, 63 942 excess deaths were observed in Germany in 2020 (25). Thus, the findings of our research showed similarities with the studies conducted in other countries.

The number of weekly deaths in Bursa showed an increasing trend after the 41st week of 2020, peaked at the 48th week, and then started to decrease (Figure 1). However, deaths due to communicable diseases increased from the 13th week and followed a plateau until the 43rd week, with a rapid increase after the 43rd week peaked at the 48th week, and then showed a decreasing trend (Figure 3). In addition, the number of excess deaths started to increase after the 11th week of 2020 in İstanbul (19, 20). The first COVID-19 death in Turkey was announced in the 12th week of 2020 (March 17, 2020). Our findings are consistent with the first COVID-19 death in the country and the studies conducted in Istanbul. 

Excess mortality during the COVID-19 pandemic varied according to countries’ strategies and the capacity of their health system. Excess mortality in Germany in 2020 was observed in two different periods between the 12th-21st and 31st-53rd weeks. The intense increase in excess deaths started at the 41st week, similar to our study, and reached its peak at the 52nd week and then started to decrease (25). In a study conducted in Stockholm, excess deaths increased rapidly at the 12th week and continued to increase between the 12th-17th weeks (16). In France, excess deaths were observed at the 11th week, peaked at the 14th week, and then trended downward  (17). In parallel with the spread of the pandemic, an increase was observed between the 15th-32nd weeks of 2020 in the USA (1). In the countries monitored within the scope of EUROMOMO, the periods in which the number of deaths started to increase were observed as two waves. The first wave between the 12th-20th weeks peaked at the 14th week, and the second wave started at the 42nd week (26). Unlike other countries, the number of deaths in the United States had not decreased to the average level in any other week since the 13th week of 2020 (6).

When the number of excess deaths in Bursa in 2020 was compared with the averages of the previous five years, an increase of 32.4% in women, 40.8% in men, and 35.0% in total were observed. Compared to the last five-year period, the proportion of males in all deaths also increased in 2020. In a study conducted in the UK during the first wave of COVID-19, the excess mortality rate in men was 1.4 times higher than in women (27). In a three-month study during the first wave in Italy, an increase of 32.1% in men and 27.7% in women was observed (8). In agreement with our findings, the other studies showed that the mortality rate in males was higher than in females during the COVID-19 pandemic. 

In our study, while the number of deaths decreased in the 0-14 age group in both genders, the P-score was highest in the 65+ age group. 18.4% of the excess deaths seen in France’s first wave (10th-22nd weeks) were among the 65+ age group. Similar to our study, a significant decrease was found in France’s 0-14 age group compared to other years (17). During the first wave in the UK, the excess mortality rate in the 75+ age group increased ten times compared to the 45-64 age group (27). It was reported that most deaths in the US state of California between March 7 and August 22 occurred among the 65+ age group (28). The finding of our study was consistent with other studies. In addition, WHO declared that people over 60 years were at a higher risk of COVID-19 (29).

Most of the excess deaths (85.3%) in Bursa in 2020 were because of communicable diseases. Many studies showed that the leading cause of excess mortality in 2020 was COVID-19. In the study surveying deaths between the 1st-33rd weeks in Switzerland and Stockholm, 85-90% of the excess deaths were identified as confirmed COVID-19 deaths (30). In another study conducted in Los Angeles, USA, the estimated number of excess pneumonia, influenza, and COVID-19-related deaths was approximately equal to the number of probable COVID-19 deaths and deaths confirmed by surveillance (31). According to a study investigating the deaths in Belgium until the end of June 2020, 107% of the excess deaths were due to COVID-19 (32). 

Our study showed that deaths due to communicable diseases in Bursa increased 76 times in 2020 compared to the average of the previous five years. Since the distribution of COVID-19 deaths by provinces was not declared in Turkey, we could not precisely know how many communicable disease-related deaths in Bursa were because of COVID-19, and it was not possible to make comparisons with other provinces. However, since there was no increase in communicable diseases other than COVID-19 in 2020, we could assume that all excess deaths from communicable diseases were COVID-19-related. 

Based on this assumption, the COVID-19 mortality rate in Bursa in 2020 could be calculated as 1464 per million, according to the threshold value from the average number of deaths of the last five years (Table 3, EM1). At the end of 2020, Bursa ranked fourth globally after Peru, San Marino and Belgium, with its COVID-19 mortality rate per million people (22). According to the official announcements of the Turkish Ministry of Health, Turkey’s COVID-19 death rate at the end of 2020 was 248 per million (22). The fact that the COVID-19 mortality rate was much higher in Bursa than in Turkey may be related to the possible underreporting of COVID-19 deaths by the Ministry of Health, as mentioned before. Claims that COVID-19 deaths are higher than official numbers worldwide are still discussed (33).

Our study is limited because the number of confirmed and possible COVID-19 deaths could not be obtained from the Provincial Health Directorate.

Given that the Turkish Ministry of Health does not share detailed data on the COVID-19 pandemic, it is almost impossible to evaluate the response to the pandemic on a regional basis and the effects of the measures taken. Reporting suspected and confirmed deaths from COVID-19 is essential to indicate the actual burden of the pandemic.

Ethical Approval: The Ethics Committee of Bursa Uludag University Faculty of Medicine Research approved the study with the decission number 2020-21/17.

Informed Consent: N.A.

Peer-review: Externally peer-reviewed

Author Contributions: Concept – K.P.; Design – K.P.; Supervision – K.P., A.T.; Materials – N.Y., N.K.Ç.; Data Collection and/or Processing – N.Y., N.K.Ç.; Analysis and/or Interpretation – K.P., N.Y., N.K.Ç., A.T.; Literature Review – K.P., N.Y., N.K.Ç., A.T.; Writer –; K.P., N.Y., N.K.Ç.; Critical Reviews – K.P., A.T.

Conflict of Interest: The authors declare no conflict of interest.

Financial Disclosure: The authors declared that this study has received no financial support. 

Show References

References

  1. Rossen LM, Branum AM, Ahmad FB, Sutton P, Anderson RN. Excess deaths associated with COVID-19, by age and race and ethnicity — United States, January 26–October 3, 2020. MMWR Morbidity and Mortality Weekly Report. 2020;69(42):1522–7. [CrossRef]
  2. Department of Health and Mental Hygiene NYC. Preliminary estimate of excess mortality during the COVID-19 outbreak —. 2020;69(19):603–5. [CrossRef]
  3. Beaney T, Clarke JM, Jain V, Golestaneh AK, Lyons G, Salman D, et al. Excess mortality: the gold standard in measuring the impact of COVID-19 worldwide? J R Soc Med. 2020;113(9):329–34. [CrossRef]
  4. Simonsen L, Clarke MJ, Williamson GD, Stroup DF, Arden NH, Schonberger LB. The impact of influenza epidemics on mortality: introducing a severity index. Am J Public Health. 1997;87(12):1944–50. [CrossRef]
  5. Collins D. S. Excess mortality from causes other than influenza and pneumonia during Influenza Epidemics. Public Health Reports. 1932;47:2159–79. [CrossRef]
  6. Excess deaths associated with COVID-19 [Internet]. Centers for Disease Control and Prevention (CDC). (cited November 12, 2020). Available from: https://www.cdc.gov/nchs/nvss/vsrr/covid19/excess_deaths.htm
  7. Vanella P, Basellini U, Lange B. Assessing excess mortality in times of pandemics based on principal component analysis of weekly mortality data – the case of COVID-19. Genus. 2021;77:16. [CrossRef]
  8. Scortichini M, Schneider dos Santos R, De’ Donato F, De Sario M, Michelozzi P, Davoli M, et al. Excess mortality during the COVID-19 outbreak in Italy: a two-stage interrupted time-series analysis. Int J Epidemiol. 2021;49(6):1909-17. [CrossRef]
  9. Excess mortality in England, week ending 26 February 2021 [Internet]. Public Health England. (cited October 14, 2021). Available from: https://fingertips.phe.org.uk/static-reports/mortality-surveillance/excess-mortality-in-england-week-ending-26-Feb-2021.html
  10. COVID-19 Information Page: General Coronavirus Table [Internet]. Republic of Turkey Ministry of Health. (cited October 14, 2021). Turkish. Available from: https://covid19.saglik.gov.tr/TR-66935/genel-koronavirus-tablosu.html
  11. Elbek O. COVID-19 outbreak and Turkey. Turk Thorac J. 2020;21(3):215–6. [CrossRef]
  12. Wang W, Xu Y, Gao R, Lu R, Han K, Wu G, et al. Detection of SARS-CoV-2 in different types of clinical specimens. JAMA. 2020;323(18):1843–4. [CrossRef]
  13. Pala K. COVID-19: Case and death notification problems in Turkey. Turk Thorac J. 2020;21(3):213–4. [CrossRef]
  14. The results of address based population registration system, 2020 [Internet]. Turkish Statistical Institute. (cited October 14, 2021). Turkish. Available from: https://data.tuik.gov.tr/Bulten/Index?p=Adrese-Dayali-Nufus-Kayit-Sistemi-Sonuclari-2020-37210
  15. Estimating Excess Mortality From COVID-19 [Internet]. Prevent Epidemics. (May 14, 2020; cited October 14, 2021). Available from: https://preventepidemics.org/covid19/resources/excess-mortality/
  16. Pimenoff VN, Elfström M, Baussano I, Björnstedt M, Dillner J. Estimating total excess mortality during a COVID-19 outbreak in Stockholm, Sweden. Clin Infect Dis. 2021;72(11):e890-2. [CrossRef]   
  17. Fouillet A, Pontais I, Caserio-Schönemann C. Excess all-cause mortality during the first wave of the COVID-19 epidemic in France, March to May 2020. Euro Surveill. 2020;25(34):2001485. [CrossRef]
  18. Dahal S, Banda JM, Bento AI, Mizumoto K, Chowell G. Characterizing all-cause excess mortality patterns during COVID-19 pandemic in Mexico. BMC Infect Dis. 2021;21(1):432. [CrossRef]
  19. Musellim B, Kul S, Ay P, Uyanusta Küçük FÇ, Dağlı E, Itil O, et al. Excess mortality during COVID-19 pandemic in İstanbul. Turk Thorac J. 2020;22(2):137–41. [CrossRef]
  20. Yardım MS, Eser S. COVID-19 pandemisi ve fazladan ölümler: İstanbul örneği. Turk J Public Health. 2020;18(Special Issue):14–24. Turkish. [CrossRef]
  21. Bursa Büyükşehir Belediye Başkanı Aktaş’tan Kovid-19 güvencesi [Internet]. Bursa Hakimiyet. (December 17, 2020; cited December 23). Turkish. Available from: https://www.bursahakimiyet.com.tr/haber/bursa-buyuksehir-belediye-baskani-aktas-tan-kovid-19-guvencesi-462042.html
  22. Ritchie H, Mathieu E, Rodés-Guirao L, Appel C, Giattino C, Ortiz-Ospina E, et al. Coronavirus Pandemic (COVID-19) [Internet]. Our World in Data. (March 5, 2020; cited October 14, 2021) Available from: https://ourworldindata.org/coronavirus
  23. “Türkiye’nin Wuhan’ı” İstanbul’da pandeminin bir yılı nasıl geçti? [Internet]. Bianet. (March 23, 2021; cited October 14, 2021). Turkish. Available from: https://m.bianet.org/bianet/saglik/241271-turkiye-nin-wuhan-i-istanbul-da-pandeminin-bir-yili-nasil-gecti
  24. Yavuz CI, Pala K. Türkiye’de COVID-19 Pandemisinin Epidemiyolojisi. In: COVID-19 Pandemisi 1. Yıl Değerlendirme Raporu. Ankara: Türk Tabipleri Birliği, 2021:5-11. Turkish.
  25. Short-term Mortality Fluctuations [Internet]. Human Mortality Database. (cited June 18, 2021). Available from: https://mpidr.shinyapps.io/stmortality/
  26. Graphs and maps [Internet]. EUROMOMO. (cited November 12, 2020). Available from: https://www.euromomo.eu/graphs-and-maps
  27. De Lusignan S, Joy M, Oke J, McGagh D, Nicholson B, Sheppard J, et al. Disparities in the excess risk of mortality in the first wave of COVID-19: Cross sectional study of the English sentinel network. J Infect. 2020;81(5):785–92. [CrossRef]
  28. Chen YH, Glymour MM, Catalano R, Fernandez A, Nguyen T, Kushel M, et al. Excess Mortality in California during the Coronavirus Disease 2019 Pandemic, March to August 2020. JAMA Intern Med. 2021;181(5):705-7. [CrossRef]
  29. COVID-19 vulnerable and high risk groups [Internet]. WHO Western Pacific. (cited October 14, 2021). Available from: https://www.who.int/westernpacific/emergencies/covid-19/information/high-risk-groups
  30. Modig K, Ahlbom A, Ebeling M. Excess mortality from COVID-19: weekly excess death rates by age and sex for Sweden and its most affected region. Eur J Public Health. 2021;31(1):17–22. [CrossRef]
  31. Traub E, T. Amoon A, Rollin-Alamillo L, Haddix M, Poortinga K, Ibrahim M, et al. Excess mortality associated with the COVID-19 pandemic – Los Angeles County, March-September 2020. J Public Health Manag Pract. 2021;27(3):233–9. [CrossRef]
  32. Molenberghs G, Faes C, Aerts J, Theeten H, Devleesschauwer B, Sierra NB, et al. Belgian COVID-19 mortality, excess deaths, number of deaths per million, and infection fatality rates (8 March – 9 May 2020). medRxiv 2020.06.20.20136234. [CrossRef]
  33. COVID-19 has caused 6.9 million deaths globally, more than double what official reports show [Internet].  Institute for Health Metrics and Evaluation. (cited October 14, 2021). Available from: http://www.healthdata.org/news-release/covid-19-has-caused-69-million-deaths-globally-more-double-what-official-reports-show