Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 RESEARCH ARTICLE    Lessons learned and lessons missed: impact of the coronavirus disease 2019 (COVID-19) pandemic on all-cause mortality in 40 industrialised countries and US states prior to mass vaccination [version 2; peer review: 2 approved] Vasilis Kontis 1, James E. Bennett1, Robbie M. Parks 2,3, Theo Rashid1, Jonathan Pearson-Stuttard1, Perviz Asaria1, Bin Zhou1, Michel Guillot4,5, Colin D. Mathers6, Young-Ho Khang7, Martin McKee 8, Majid Ezzati 1,9 1MRC Centre for Environment and Health, School of Public Health, Imperial College London, London, UK 2The Earth Institute, Columbia University, New York, NY, USA 3Department of Environmental Health Sciences, Mailman School of Public Health, Columbia University, New York, NY, USA 4Population Studies Center, Department of Sociology, University of Pennsylvania, Philadelphia, PA, USA 5French Institute for Demographic Studies (INED), Paris, France 6Independent Researcher, Geneva, Switzerland 7Institute of Health Policy and Management, Seoul National University, Seoul, South Korea 8Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, London, UK 9Regional Institute for Population Studies, University of Ghana, Legon, Ghana v2 First published: 18 Oct 2021, 6:279 Open Peer Review https://doi.org/10.12688/wellcomeopenres.17253.1 Latest published: 15 Feb 2022, 6:279 https://doi.org/10.12688/wellcomeopenres.17253.2 Approval Status 1 2 Abstract Background: Industrialised countries had varied responses to the version 2 COVID-19 pandemic, which may lead to different death tolls from (revision) COVID-19 and other diseases. view view15 Feb 2022 Methods: We applied an ensemble of 16 Bayesian probabilistic models to vital statistics data to estimate the number of weekly deaths if the pandemic had not occurred for 40 industrialised version 1 18 Oct 2021 view view countries and US states from mid-February 2020 through mid- February 2021. We subtracted these estimates from the actual number of deaths to calculate the impacts of the pandemic on all- 1. Rajeev Gupta , Rajasthan University of cause mortality. Health Sciences, Jaipur, India Results: Over this year, there were 1,410,300 (95% credible interval 1,267,600-1,579,200) excess deaths in these countries, equivalent to a Eternal Heart Care Centre and Research 15% (14-17) increase, and 141 (127-158) additional deaths per 100,000 Institute, Jaipur, India people. In Iceland, Australia and New Zealand, mortality was lower than would be expected in the absence of the pandemic, while South 2. Virgilio Gómez-Rubio , Universidad de Korea and Norway experienced no detectable change. The USA, Castilla-La Mancha, Ciudad Real, Spain Czechia, Slovakia and Poland experienced >20% higher mortality. Within the USA, Hawaii experienced no detectable change in mortality Any reports and responses or comments on the and Maine a 5% increase, contrasting with New Jersey, Arizona,   Page 1 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Mississippi, Texas, California, Louisiana and New York which experienced >25% higher mortality. Mid-February to the end of May article can be found at the end of the article. 2020 accounted for over half of excess deaths in Scotland, Spain, England and Wales, Canada, Sweden, Belgium, the Netherlands and Cyprus, whereas mid-September 2020 to mid-February 2021 accounted for >90% of excess deaths in Bulgaria, Croatia, Czechia, Hungary, Latvia, Montenegro, Poland, Slovakia and Slovenia. In USA, excess deaths in the northeast were driven mainly by the first wave, in southern and southwestern states by the summer wave, and in the northern plains by the post-September period. Conclusions: Prior to widespread vaccine-acquired immunity, minimising the overall death toll of the pandemic requires policies and non-pharmaceutical interventions that delay and reduce infections, effective treatments for infected patients, and mechanisms to continue routine health care. Keywords Excess mortality, Covid-19, SARS-CoV-2, Bayesian ensemble modelling, autoregressive models, uncertainty. This article is included in the Coronavirus (COVID-19) collection. Corresponding author: Majid Ezzati (majid.ezzati@imperial.ac.uk) Author roles: Kontis V: Conceptualization, Data Curation, Formal Analysis, Methodology, Software, Validation, Visualization, Writing – Original Draft Preparation; Bennett JE: Conceptualization, Data Curation, Methodology, Visualization; Parks RM: Data Curation, Methodology; Rashid T: Data Curation, Methodology; Pearson-Stuttard J: Writing – Review & Editing; Asaria P: Writing – Review & Editing; Zhou B: Data Curation, Writing – Review & Editing; Guillot M: Methodology, Writing – Review & Editing; Mathers CD: Data Curation, Writing – Review & Editing; Khang YH: Data Curation, Writing – Review & Editing; McKee M: Writing – Review & Editing; Ezzati M: Conceptualization, Funding Acquisition, Methodology, Supervision, Visualization, Writing – Original Draft Preparation Competing interests: ME reports a charitable grant from the AstraZeneca Young Health Programme, outside the submitted work. JP-S is vice-chair of the Royal Society for Public Health and reports personal fees from Novo Nordisk A/S and Lane, Clark & Peacock LLP, outside of the submitted work. Grant information: The development of methodology for estimating the impact of pandemic as an extreme event was supported by the Wellcome Trust [209376; Pathways to Equitable Healthy Cities]. Work on the US mortality data was partially supported by a grant from the US Environmental Protection Agency (EPA), as part of the Center for Clean Air Climate Solution [assistance agreement no. R835873]. This article has not been formally reviewed by the EPA. The views expressed in this document are solely those of the authors and do not necessarily reflect those of the EPA. The EPA does not endorse any products or commercial services mentioned in this publication. Work on the UK mortality data was partially supported by the British Heart Foundation [RE/18/4/34215; Centre of Research Excellence grant]. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Copyright: © 2022 Kontis V et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Kontis V, Bennett JE, Parks RM et al. Lessons learned and lessons missed: impact of the coronavirus disease 2019 (COVID-19) pandemic on all-cause mortality in 40 industrialised countries and US states prior to mass vaccination [version 2; peer review: 2 approved] Wellcome Open Research 2022, 6:279 https://doi.org/10.12688/wellcomeopenres.17253.2 First published: 18 Oct 2021, 6:279 https://doi.org/10.12688/wellcomeopenres.17253.1   Page 2 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 • W e could access up-to-date weekly data on all-  RE VI SE D  Amendments from Version 1 cause mortality divided by age group and/or sex that Based on the helpful comments from reviewers, we have added extended through February 2021. results on US states in the abstract, reorganised the text, included additional citations, and done a sensitivity analysis on • T he time series of data went back at least to the begin- how different models are weighted. We have also updated the ning of 2016 so that model parameters could be reliably section Comparison with other estimates to include the most estimated. For countries with longer time series, we used recently published results from other sources. data starting in 2010. Any further responses from the reviewers can be found at  the end of the article The 40 countries in our analysis were divided into five geographical regions: the Pacific (Australia, New Zealand, South Korea), the Americas (Canada, Chile, the USA), Central and Eastern Europe (Austria, Bulgaria, Croatia, Czechia, Estonia, Introduction Hungary, Latvia, Lithuania, Montenegro, Poland, Romania, Many industrialised countries experienced a rise in all-cause Serbia, Slovakia, Slovenia), Southwestern Europe (Cyprus, mortality in the first wave of the coronavirus disease 2019 France, Greece, Italy, Malta, Portugal, Spain), Northwestern (COVID-19) pandemic, while others avoided any excess Europe (Belgium, England and Wales, Germany, Luxembourg, deaths1. These excess deaths were due to infection with severe the Netherlands, Northern Ireland, Scotland, Switzerland) and acute respiratory syndrome coronavirus 2 (SARS-CoV-2), Nordic (Denmark, Finland, Iceland, Norway, Sweden). In addi- delays and disruptions in the provision and use of healthcare tion to national estimates, we separately estimated excess for other diseases, loss of jobs and income, disruptions of social deaths for all 50 US states and the District of Columbia. networks and support, and changes in nutrition, drug and Some US states are larger than most countries included in our alcohol use, transportation, crime, and violence2,3. analysis (e.g., California’s population of ~40 million is larger than those of 33 of the countries in the analysis), and the Decline in infections following initial lockdowns and other extent and temporal dynamics of the pandemic were hetero- restrictions, and advances in knowledge about the SARS-CoV-2 geneous across states due to their relative autonomy in policy transmission and infection, presented a window of opportu- formulation and implementation. nity for countries to implement pandemic control measures and strengthen health and social care provision that would minimise The sources of population and mortality data are provided the impacts of subsequent waves4–8. Comparative analysis of in Table 1. We calculated weekly population through inter- excess deaths prior to mass vaccination against COVID-19 polation of yearly population, consistent with the approach helps understand how effectively these measures were imple- taken by national statistical offices for intra-annual population mented and how resilient the health and social care sys- calculation10. Population for 2020 and 2021, where not available, tem was in each country. We quantified the weekly mortality was obtained through linear extrapolation from the last five impacts of the first year of the COVID-19 pandemic, from mid- years. We obtained data on temperature from ERA511, which February 2020 to mid-February 2021, in 40 industrialised uses data from global in situ and satellite measurements to countries, listed below. We used this period because mortality generate a worldwide meteorological dataset, with full space due to the pandemic was negligible before mid-February and time coverage over our analysis period. We used gridded 20201, and vaccination rates against SARS-CoV-2 were still temperature estimates measured four times daily at a resolution relatively low before mid-February 2021 in these countries of 30 km to generate weekly temperatures for each first-level (no more than 4% of the population had received both administrative region, and gridded population data to generate doses in any of these countries, as per Our World in Data9). population estimates by first-level administrative region in After mid-February 2021, the effect of vaccines on mortality each country. We weighted weekly temperature by population was expected to appear in some countries, which should of each first-level administrative region to create national be subject to a distinct analysis. level weekly temperature summaries. Methods Statistical methods Data sources We used a probabilistic model averaging approach to estimate We included industrialised countries with complete or what deaths were expected to be between mid-February 2020 near-complete registration of deaths in our analysis if: and mid-February 2021 had the pandemic not occurred, and • T heir total population in 2020 was more than 100,000. compared these estimates with actual deaths from all causes in We excluded countries (e.g., Liechtenstein) with data each country. The analytical method was designed to enhance but with smaller populations because, in many weeks, comparison across countries and over time, and account for the number of deaths would be small or zero. This medium-long-term secular trends in mortality, the potential would, in turn, lead to either large uncertainty that would dependency of death rates in each week on those in preceding make it hard to differentiate between those places with week(s) and in each year on those in preceding year(s), and fac- and without an effect or unstable estimates because the tors that affect mortality including seasonality, temperature and model is fitted to many weeks with zero deaths. public holidays. Page 3 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Table 1. Sources of data on deaths and population. Data sources for deaths  Start of time  Sex-specific analysis (see Analysis age groups (see and population series Methods for details) Methods for details) Australia ABS1, UN2 29/12/2014 Y 0-44, 45-64, 65+ Austria Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Belgium Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Bulgaria Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Canada StatCan5,6 09/01/2010 Y 0-44, 45-64, 65+ Chile MINSAL7, UN2 01/01/2016 Y 0-44, 45-64, 65+ Croatia Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Cyprus Eurostat3,4 29/12/2014 Y 0-64, 65+ Czechia Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Denmark Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ England and ONS9,10 02/01/2010 Y 0-44, 45-64, 65+ Wales8 Estonia Eurostat3,4 04/01/2010 Y 0-64, 65+ Finland Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ France Eurostat3,4 31/12/2012 Y 0-44, 45-64, 65+ Germany Destatis11, Eurostat4 04/01/2016 Y 0-44, 45-64, 65+ Greece Eurostat3,4 29/12/2014 Y 0-44, 45-64, 65+ Hungary Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Iceland Eurostat3,4 04/01/2010 N All ages Italy Eurostat3,4 03/01/2011 Y 0-44, 45-64, 65+ Latvia Eurostat3,4 04/01/2010 Y 0-64, 65+ Lithuania Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Luxembourg Eurostat3,4 04/01/2010 Y 0-64, 65+ Malta Eurostat3,4 03/01/2011 N All ages Montenegro Eurostat3,4 04/01/2010 Y 0-64, 65+ Netherlands Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ New Zealand Stats NZ12, UN2 02/01/2011 Y 0-64, 65+ Northern NISRA13, Eurostat4 01/01/2011 N 0-64, 65+ Ireland8 Norway Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Poland Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Portugal Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Romania Eurostat3,4 29/12/2014 Y 0-44, 45-64, 65+ Scotland8 NRS14, ONS10 04/01/2010 Y 0-44, 45-64, 65+ Serbia Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Slovakia Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Page 4 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Data sources for deaths  Start of time  Sex-specific analysis (see Analysis age groups (see and population series Methods for details) Methods for details) Slovenia Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ South Korea KOSIS15 03/01/2010 N 0-64, 65+ Spain Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Sweden Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ Switzerland Eurostat3,4 04/01/2010 Y 0-44, 45-64, 65+ USA CDC16,17 04/01/2015 N 0-44, 45-64, 65+18 1 https://www.abs.gov.au/statistics/health/causes-death/provisional-mortality-statistics/latest-release 2 https://population.un.org/wpp 3 https://ec.europa.eu/eurostat/data/database (table demo_r_mwk_05). Deaths with unknown age (0.03% of all deaths) were distributed across age groups proportional to the overall distribution of deaths for each year and month. 4 https://ec.europa.eu/eurostat/data/database (table demo_pjangroup) 5 https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310076801. Death counts rounded to a neighbouring multiple of 5. There were no data for Yukon from 2017 to 2021 (before 2017, there were <10 deaths per week in Yukon). 6 https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000501 7 https://deis.minsal.cl/#datosabiertos. Deaths with unknown age and/or sex (0.02% of all deaths) were distributed across age groups and sexes proportional to the overall distribution of deaths for each year and month. 8 Data for the constituent nations in the UK are provided separately by NISRA for Northern Ireland, NRS for Scotland and ONS for England and Wales. These datasets use different reporting week definitions and could therefore not be combined into a single time series for the UK. 9 https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/ weeklyprovisionalfiguresondeathsregisteredinenglandandwales 10 https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/datasets/ populationestimatesforukenglandandwalesscotlandandnorthernireland 11 https://www.destatis.de/DE/Themen/Gesellschaft-Umwelt/Bevoelkerung/Sterbefaelle-Lebenserwartung/Tabellen/sonderauswertung- sterbefaelle.html 12 https://www.stats.govt.nz/experimental/covid-19-data-portal 13 https://www.nisra.gov.uk/publications/historical-weekly-deaths-data and https://www.nisra.gov.uk/publications/weekly-death-statistics- northern-ireland-2021 14 https://www.nrscotland.gov.uk/statistics-and-data/statistics/statistics-by-theme/vital-events/general-publications/weekly-and-monthly- data-on-births-and-deaths/deaths-involving-coronavirus-covid-19-in-scotland/related-statistics 15 https://kosis.kr/covid_eng/statistics_excessdeath.do and https://mdis.kostat.go.kr/index.do 16 https://data.cdc.gov/NCHS/Weekly-counts-of-deaths-by-jurisdiction-and-age-gr/y5bj-9g5w. We used deaths adjusted for completeness by the CDC which account for potential underreporting in the most recent weeks. The adjustment methods are described at https:// www.cdc.gov/nchs/nvss/vsrr/covid19/tech_notes.htm. 17 https://www.cdc.gov/nchs/nvss/bridged_race/data_documentation.htm 18 When analysing individual states, we merged 0–44 and 45–64 age groups into a single age group 0-64 years for Alaska, Delaware, DC, Hawaii, Idaho, Maine, Montana, Nebraska, New Hampshire, North Dakota, Rhode Island, South Dakota, Vermont West Virginia and Wyoming for reasons described in Methods. The total mortality impact of the COVID-19 pandemic is the behave in the short-term (week to week) and medium-term difference between the observed number of deaths from all (year to year), as follows: causes of death and the number of deaths had the pandemic not • F irst, death rates may have a medium-to-long-term occurred, which is not directly measurable. The most com- trend13 that would lead to a lower or higher mortality mon approach to calculating the number of deaths had the pan- in 2020-2021 compared to earlier years. Therefore, all demic not occurred has been to use the average number of deaths models included a linear trend term over weekly death over previous years, e.g., the most recent five years, for the rates. corresponding week or month when the comparison is made. This approach however does not take into account long- and • S econd, death rates have a seasonal pattern14–17. We short-term trends in mortality or time-varying factors like included weekly random intercepts for each week of the temperature that are largely external to the pandemic, but year. To account for the fact that seasonal patterns “repeat” also affect death rates. (i.e., late December and early January are seasonally similar) we used a seasonal structure18,19 for the random We developed an ensemble of 16 Bayesian mortality projec- intercepts. The seasonal structure allows the magnitude tion models that each make an estimate of weekly death rates of the random intercepts to vary over time, and implic- that would have been expected if the COVID-19 pandemic had itly incorporates time-varying factors such as annual not occurred12. We used multiple models because there is inher- fluctuations in flu season. ent uncertainty in the choice of model that best predicts death rates in the absence of pandemic. These models were formulated • T hird, death rates in each week may be related to rates in to incorporate features of weekly death rates, and how they preceding week(s), due to short-term phenomena such Page 5 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 as severity of the flu season. We formulated four sets The term α denotes the overall intercept and α 0 holiday(week) of models to account for this relationship. The weekly is the holiday intercept, applied to weeks with a holiday. random intercepts in these models had a first, second, For example, if a week includes the 25th of December then fourth or eighth order autoregressive structure18,19. The α = α . For weeks that did not contain a holi-holiday(week) Christmas higher-order autoregressive models allow death rates day, this term did not appear in the above expression. All in any week to be informed by those in a progressively intercepts were assigned 𝒩(0,1000) priors. The term β ⋅ week larger number of preceding weeks. Further, trends not represents the linear time trend. The coefficient β was also picked up by the linear or seasonal terms would be assigned a 𝒩(0,1000) prior. captured by these autoregressive terms. The models used different orders (first, second, fourth or • F ourth and additionally, mortality in one year may depend (i ) eighth) of the autoregressive term ζ with the superscript i on mortality in the previous year, in a different way week for each month, because phenomena such as seasonal denoting the order for weekly mortality patterns. The first- (1) flu may lead to longer-term dependencies in mortality. order autoregressive term is defined as ζweek ~ 𝒩 (1) 2 (ϕ ⋅ ζ ) week−1,σ ζ To allow for this possibility, we used two sets of where the parameter φ lies between -1 and 1 and captures models, with and without a first order autoregressive the degree of association between the number of deaths in term over years for each month. each week and the preceding week. Hyperpriors are placed on the parameters ϰ = log 2((1–φ2)/σ ζ ) and ϰ = log ((1+φ)/(1–φ)) • 1 2Fifth, beyond having a seasonal pattern, death rates which were assigned logGamma(0.001,0.001) and 𝒩(0,1) depend on temperature, and specifically on whether distributions respectively. Similarly, an ith order autoregres- temperature is higher or lower than its long-term norm sive term is given by ζ( i ) ( i ) ( i )week = ϕ1 ⋅ζweek−1+ ... + ϕ i ⋅ζweek−i+ εduring a particular time of year20–25. The effect of week temperature on mortality varies throughout the year, with –1 < ϕ < 1. The parametrisation of these models was based j and may be in opposite directions for different times on the partial autocorrelation function of the sequence ϕ 26. j of year. We used two sets of models, one without tem- month perature and one with a weekly term for temperature The term ηyear is an autoregressive term of order 1 over years anomaly, defined as deviation of weekly temperature and independent across months, indexed to the month and year from the local average weekly temperature over the to which each particular week belongs. For each month, the month (1) entire analysis period. autoregressive prior for ηyear was the same as that for ζweek described above. As described above, this term appeared in half • F inally, death rates may be different around major holi- of our models. days such as Christmas and New Year either because of changes in human activities and behaviour or, for the The term θ captures seasonality in mortality trends with countries whose data are registration based, because week a period of 52 weeks. The sums of every 52 consecutive of delays in registration. We included effects (as fixed terms θ + θ + ... + θ were modelled as independ- intercepts) for the weeks containing Christmas and New week week+1 week+51 2 ent Gaussian with zero mean and variance σ . We used a Year in all countries. For England and Wales, Scotland θ 2 logGamma(0.001,0.001) prior on the log precision log(1/σ ). and Northern Ireland, we also included effects for the θ Each week is assigned an index between 1 and 52 depend- week containing other public holidays, because reported ing on which week of the current year it is (the incomplete death rates in weeks that contain a holiday were dif- week 53 is mapped to either index 1 or 52 depending on ferent from other weeks. This term was tested but not whether it has greater overlap with week 52 of the current year included for other countries because the effect was or week 1 of the following year). negligible. These choices led to an ensemble of 16 Bayesian models The effect of temperature anomaly on death rates is cap- (2 yearly autoregressive options × 4 weekly autoregressive tured by the two terms γ and ν . The term γ⋅temperature week of year options × 2 temperature anomaly options). The ensemble of anomaly is the overall association between (log-transformed) week models is shown in Table 2. In each model, the number of death rates and temperature anomaly in a week. The term weekly deaths follows a Poisson distribution: ν ⋅temperature anomaly captures deviations from week of year week the overall association for each week of the year. It consists of 52 terms with an independent and identically distributed deathsweek ~ Poisson(death rateweek ⋅ populationweek). prior defined via ν ~ 𝒩(0,σ2ν), and log-precision week of year 2 log(1/σν) ~ logGamma(0.001,0.001).Log-transformed death rates were modelled as a sum of components described above: Finally, the term ε is a zero-mean term that accounts for addi- week (i ) month tional variability. It is assigned an independent and identically log (death rateweek ) = α + α ζ0 holiday(week) +β ⋅ week + week+ηyear distributed prior ε ~ 𝒩(0, σ2ε), and a logGamma(0.001,0.001) week + θweek + (γ + νweek of year ) ⋅ temperature anomaly 2 week + εweek prior was placed on the log precision log(1/σε). The Page 6 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Table 2. Combination of terms used in each of the 16 models for estimating number of weekly deaths that would be  expected had the pandemic not occurred. See Methods for an explanation of each term. Model  Global  Time slope Non-linear  Seasonal  Non-linear  Temperature anomaly  number intercepts1 (autoregressive) term (autoregressive) month- terms term over weeks specific term over years 1 α0 + α (1) holiday(week) β·week ζweek θweek - - 2 α0 + αholiday(week) β·week ζ (1) week θweek - (γ + vweek of year) ·temperature anomalyweek 3 α + α (1) month0 holiday(week) β·week ζweek θweek η -year 4 α + α β·week ζ(1)week θ ηmonth0 holiday(week) week (γ + vweek of year) ·temperature year anomalyweek 5 α + α (2)0 holiday(week) β·week ζweek θweek - - 6 α0 + α (2) holiday(week) β·week ζweek θweek - (γ + vweek of year) ·temperature anomalyweek 7 α (2)0 + αholiday(week) β·week ζweek θ ηmonthweek -year 8 α0 + α (2) holiday(week) β·week ζweek θ ηmonthweek (γ + vyear week of year) ·temperature anomalyweek 9 α0 + α (4) holiday(week) β·week ζweek θweek - - 10 α + α β·week ζ(4)0 holiday(week) θweek - (γ + vweek week of year) ·temperature anomalyweek 11 α0 + αholiday(week) β·week ζ (4) week θ monthweek η -year 12 α0 + α (4) month holiday(week) β·week ζweek θweek η (γ + vyear week of year) ·temperature anomalyweek 13 α + α (8)0 holiday(week) β·week ζ θ - -week week 14 α (8)0 + αholiday(week) β·week ζ θweek week - (γ + vweek of year) ·temperature anomalyweek 15 α + α (8) month0 holiday(week) β·week ζ θweek week η -year 16 α + α β·week ζ(8) month0 holiday(week) θweek η (γ + vweek year week of year) ·temperature anomalyweek 1 Due to the short duration of the time series, the holiday term was not identifiable for Chile (in the presence of the seasonal term) and was therefore not included. (i ) components α ,α , β⋅week, θ , ε and ζ (for prediction periods assumes that the number of deaths that are 0 holiday(week) week week week autoregressive order i = 1,2,4 or 8) appear in the expression for directly or indirectly related to the COVID-19 pandemic was log(death rate ) in all models. The remaining components negligible through mid-February 2020 in these countries 1, week appear in some models only. Table 2 shows the terms included and separates the training data from subsequent weeks when in each of the 16 models in the ensemble. impacts may have appeared. All models were fitted using integrated nested Laplace approx- We used data on weekly deaths from the start of time series imation (INLA)27, implemented in the R-INLA software through mid-February 2020 to estimate the parameters of each (version 20.03). We used a model averaging approach to com- model, which were then used to predict death rates for the sub- bine the predictions from the 16 models in the ensemble28,29. Spe- sequent 52 weeks as estimates of the counterfactual death cifically, we took 2,000 draws from the posterior distribution of rates if the pandemic had not occurred. These were then com- predicted deaths under each of the 16 models, and pooled the pared to reported deaths to calculate excess mortality due to 32,000 draws to obtain the posterior distribution of deaths if the the pandemic. For the projection period, we used recorded COVID-19 pandemic had not taken place. This approach gen- temperature so that our projections take into consideration erates a distribution of estimates that has equal samples from actual temperature in 2020-2021. This choice of training and each model in the ensemble, and hence incorporates both the Page 7 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 uncertainty of estimates from each model and the uncertainty in model so that the uncertainty of the estimates is correctly the choice of model. The reported credible intervals represent the reported. 2.5th and 97.5th percentiles of the resultant posterior distribu- tion of the draws from the entire ensemble. We report the We report results for the entire year, as well as for three number of excess deaths, excess deaths per 100,000 people, non-overlapping periods: the first wave of the pandemic and relative (percent) increase in deaths together with their (from mid-February 2020 through end of May), the (northern corresponding 95% credible intervals. For the purpose of report- hemisphere) summer period (from beginning of June to mid- ing, we rounded results on number of deaths that are 1,000 or September 2020) and subsequent wave(s) (from mid-September more to the nearest hundred to avoid giving a false sense of 2020, when schools normally open in the northern hemisphere, precision in the presence of uncertainty; results less than 1,000 to mid-February 2021). were rounded to the nearest ten. We also report the posterior probability that an estimated increase (or decrease) in deaths Validation of no-pandemic counterfactual weekly corresponds to a true increase (or decrease). Posterior probabil- deaths ity represents the inherent uncertainty in how many deaths would We tested how well our model ensemble estimates the number have occurred in the absence of the pandemic. In a country and of deaths expected had the pandemic not occurred by withhold- week in which the actual number of deaths is the same as the ing data for 52 weeks starting from mid-February (i.e., the same posterior median of the number expected in a no-pandemic coun- projection period as done for 2020–2021) for an earlier year terfactual, an increase in deaths is statistically indistinguishable and using the preceding time series of data to train the models. from a decrease; in such a situation, there is a 50% posterior In other words, we created a situation akin to 2020–2021 for probability of an increase and a 50% posterior probability of a an earlier year. We then projected death rates for the weeks decrease. Where the entire posterior distribution of the number with withheld data, and evaluated how well the model ensemble of deaths expected without the pandemic is smaller than the projections reproduced the known-but-withheld death rates. We actual number of deaths, there is a ~100% posterior probabil- repeated this for three different periods: 2017–2018 (i.e., train ity of an increase and a ~0% posterior probability of a decrease model using data from January 2010 to mid-February 2017 and and vice versa. For most countries and weeks, the posterior test for the subsequent 52 weeks), 2018–2019 (i.e., train model distribution of the number of deaths expected without the pan- using data from January 2010 to mid-February 2018 and test demic covers the observed number, but there is asymmetry in for the subsequent 52 weeks), and 2019–2020 (i.e., train model terms of whether much of the distribution is smaller or larger using data from January 2010 to mid-February 2019 and test for than the observed number. In such cases, there would be uneven the subsequent 52 weeks). We performed these tests for each posterior probabilities of an increase versus decrease in deaths, country using data for both sexes and all ages. We report the with the two summing to 100% (for example, 80% and 20%). projection error (which measures systematic bias) and absolute Posterior probabilities more distant from 50%, toward either 0% projection error (which measures any deviation from the data). or 100%, indicate more certainty. We also evaluated the sensi- Additionally, we report coverage of the projection uncertainty; tivity of our results to how the different models are weighted. if projected death rates and their uncertainties are well esti- Specifically, in the sensitivity analysis, the number of draws mated, the estimated 95% credible intervals should cover 95% from each model was inversely proportional to the absolute error of the withheld data. of prediction in the validation analyses described below. The results of the sensitivity analysis were virtually identical to those The results of model validation (Table 3) show that the esti- with equal draws, with weekly median excess deaths estimates mates of how many deaths would be expected had the pandemic differing by up to 2.4% for individual countries, and by 0.1% not occurred from the Bayesian model ensemble were unbiased, when averaged across all countries and weeks. with mean relative projection errors of 1.5% (between 0.5% and 2.2% in different years). The mean relative absolute error We did all analyses separately by sex and age group (0–44 years, was between 8.0% and 8.7% in different years. 95% coverage, 45–64 years, 65+ years) for countries with 2020 popula- which measures how well the posterior distributions of projected tion of at least two million, where age- and sex-specific data deaths coincide with withheld data was 96% for all years, were available (Table 1). For countries with 2020 popula- which shows that the posterior distribution is well estimated. tion less than 2 million, we did our analyses for two age groups (0–64 years and 65+ years) because, in many weeks, the number An earlier version of this manuscript can be found on medRxiv of deaths in the age group 0–44 was small or zero, which (doi: https://doi.org/10.1101/2021.07.12.21260387). would have led to either large uncertainty or unstable esti- mates. For the same reason, for countries with population under Results 500,000 (Iceland and Malta), we did our analyses for both sexes Excess mortality between mid-February 2020 and and all age groups combined. Models were also run for all mid-February 2021 ages and both sexes combined; the posterior medians of resultant Taken over the entire year, both sexes and all ages, an esti- estimates were nearly identical to the sum of the age-sex-specific mated 1,410,300 (95% credible interval 1,267,600–1,579,200) ones, with a mean relative difference of 0.2%, ranging from more people died in these 40 countries than would have been -1.7% to 1.1%. For this reason, in figures and tables that are for expected had the pandemic not taken place. This is equiva- all ages and both sexes, we report results from the combined lent to 141 (127–158) additional deaths per 100,000 people Page 8 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Table 3. Results of the external predictive validity (out-of-sample validation) of the estimated no-pandemic counterfactual weekly deaths from the Bayesian model  ensemble.  Each number represents the total error over the validation period, averaged across countries. Validation  Projection error  Absolute projection  Percent covered by  year (relative projection error (relative absolute 95% credible interval error) projection error) 2017 1,893 (1.8%) 9,488 (8.5%) 97% 2018 1,107 (0.5%) 9,455 (8.7%) 94% 2019 3,306 (2.2%) 8,645 (8.0%) 98% All three years 2,102 (1.5%) 9,196 (8.4%) 96% * Australia, Chile, Cyprus, Germany, Greece, Romania and USA were not used for validation analysis because they had shorter time series. Hence leaving out the last three years of data would leave a time series that was too short for estimating model parameters. and a 15% (14–17) increase in deaths over this period in all of increase in deaths >99%. There was as much variation in excess these countries combined. The number of deaths assigned to mortality across US states as across the 40 countries together, COVID-19 in these countries over the same period was 1,256,861, with Hawaii having experienced the same level of mortal- which is 89% of the excess all-cause death toll (Table 4). The ity as would have been expected without the pandemic, Maine a number of excess deaths were largest in the USA (623,100; 5% increase, and, at the other extreme, New Jersey, Arizona, 521,200–750,700), followed by Italy (118,800; 88,500-149,300) Mississippi, Texas, California, Louisiana and New York at and England and Wales (102,100; 75,300–128,600) (Figure 1 least 25% higher mortality over this year (Figure 4). and Table 4). Within the USA, California (71,800; 64,100–79,500) and Texas (57,400; 48,100-67,200) experienced the largest Dynamics of excess mortality number of excess deaths, about the same as excess deaths in There was substantial heterogeneity across countries in terms Spain and France, respectively (Figure 2). of the patterns and dynamics of excess mortality over time (Figure 5 and Figure 6). Some countries in Central and Eastern In Iceland, Australia and New Zealand, mortality was 3–6% Europe – Bulgaria, Lithuania, Poland, Romania, Serbia and lower over this period than what would be expected if the Montenegro – had no or little excess mortality in the first wave pandemic had not occurred, with posterior probabilities of of the pandemic (mid-February 2020 to end of May 2020), but the estimated decrease being a true decrease ranging 82–94% experienced between 5% and 13% increase in mortality during (Figure 3). South Korea and Norway experienced no detectable the (northern hemisphere) summer (June 2020 to mid-September change in mortality (54% and 74% probability of an increase 2020; Figure 7A). In contrast, some countries with medium respectively, with posterior median estimated increases <2%), or high levels of excess mortality in the first wave returned and Finland, Greece, Cyprus and Denmark experienced increases to death rates in the summer that were about the same as the of 2–5% (Figure 3A), with posterior probabilities that these no-pandemic baseline (England and Wales, Belgium, Scotland, changes represent an increase in death ranging from 84% Northern Ireland, Sweden, Netherlands, France, Switzerland, to 97%. At the other extreme, the populations of the USA, Luxembourg and Cyprus) or only slightly higher than this Czechia, Slovakia and Poland experienced at least 20% higher baseline (Canada, Italy and Spain). Portugal and the USA expe- mortality over these 52 weeks than they would have had the rienced a similar increase in mortality over the summer – 10% pandemic not occurred; the increase was between 15% and (1–21) and 17% (12–24), respectively – to what they had in the 20% in England and Wales, Italy, Portugal, Spain, Romania, first wave. During the same period, Australia, New Zealand and Slovenia, Lithuania, Bulgaria, Chile, Belgium and Switzerland; Iceland had a mortality deficit compared to levels that would the posterior probabilities that these countries experienced have been expected without a pandemic. In Australia and New an increase in deaths were >99%. Because baseline mortality Zealand, which were in winter season in this period, this reduc- (i.e., death rates expected without the pandemic) varied across tion has been attributed to fewer deaths from seasonal flu due countries, the ordering of countries in terms of excess deaths to reduced contact among people30–33. Chile, the other south- per 100,000 people (Figure 3B) differed from the ranking ern hemisphere country in our analysis, had 12% (8–17) higher of percent increase. Bulgaria, Romania, Lithuania, Czechia, mortality in the first wave, followed by an even larger increase Poland, Slovakia and Portugal experienced more than 200 of 21% (15–26) during the (southern hemisphere) winter period. excess deaths per 100,000 people and Italy, USA, England and Wales, Slovenia, Spain, Croatia, Belgium and Montenegro The subsequent wave(s) of the pandemic (mid-September 2020 between 150 and 200, all with posterior probabilities of an to mid-February 2021) saw yet more changes in excess deaths Page 9 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Table 4. Number of excess deaths from any cause and  Country Number of excess deaths  Number  deaths assigned to coronavirus disease 2019 (COVID-19) due (95% credible interval) of deaths  from mid-February 2020 to mid-February 2021, by country.  assigned to  Excess deaths ≥1,000 are rounded to the nearest hundred and COVID-19 as excess deaths <1,000 to the nearest ten. Deaths assigned to underlying  COVID-19 were taken directly from the cited sources and not cause1 rounded. Portugal 20,700 (14,100 to 27,200) 15,962 Country Number of excess deaths  Number  Romania 45,600 (30,800 to 61,500) 19,894 (95% credible interval) of deaths  assigned to  Scotland 7,100 (3,600 to 10,600) 9,355 COVID-19 as underlying  Serbia 10,300 (5,600 to 14,800) 4,337 cause1 Slovakia 11,300 (8,200 to 14,200) 6,671 Australia -4,090 (-11,590 to 2,300) 909 Slovenia 3,600 (2,700 to 4,400) 4,057 Austria 9,100 (4,900 to 13,200) 8,385 South Korea 560 (-10,870 to 11,000) 1,562 Belgium 17,900 (12,300 to 23,600) 22,077 Spain 76,100 (52,400 to 100,100) 67,636 Bulgaria 18,200 (12,800 to 23,500) 9,854 Sweden 9,900 (6,400 to 13,700) 12,914 Canada 19,800 (8,800 to 31,300) 21,723 Switzerland 10,200 (7,100 to 13,300) 9,174 Chile 18,400 (15,200 to 21,600) 20,126 USA 623,100 (521,200 to 529,070 Croatia 6,400 (3,300 to 9,400) 5,449 750,700) 1 Data are from Office for National Statistics for England and Cyprus 140 (-520 to 840) 232 Wales (https://www.ons.gov.uk/ peoplepopulationandcommunity/ birthsdeathsandmarriages/deaths/datasets/ Czechia 24,000 (18,500 to 29,300) 19,777 weeklyprovisionalfiguresondeathsregisteredinenglandandwales), NRS for Denmark 2,400 (-170 to 5,100) 2,343 Scotland (https://www.nrscotland.gov.uk/covid19stats), NISRA for Northern Ireland (https://www.nisra.gov.uk/system/files/statistics/Weekly_Deaths%20- England and Wales 102,100 (75,300 to 128,600) 128,077 %20w%20e%203rd%20September%202021.XLSX) and the European Centre for Disease Prevention and Control (ECDC) for other countries (https:// Estonia 790 (140 to 1,500) 535 opendata.ecdc.europa.eu/covid19/nationalcasedeath/csv; accessed on 21 September 2021). Finland 1,200 (-450 to 2,800) 756 France 62,700 (33,200 to 95,100) 84,306 patterns across countries. While New Zealand, Australia, Iceland, Germany 64,100 (-1,870 to 135,400) 67,903 Finland, Norway, Cyprus and South Korea remained resilient Greece 3,700 (-4,180 to 11,000) 6,297 to the rise in mortality (i.e., no or <2% increase in mortal- ity compared to the no-pandemic baseline), many countries Hungary 14,300 (7,500 to 21,300) 14,347 in Europe, especially in Central Europe, experienced a rise in Iceland -140 (-340 to 40) 29 mortality compared to the no-pandemic baseline: by >40% in Slovakia, Czechia and Poland, and by 20–40% in England and Italy 118,800 (88,500 to 149,300) 95,718 Wales, Italy, Austria, Hungary, Montenegro, Croatia, Portugal, Latvia 2,200 (1,100 to 3,400) 1,542 Switzerland, Romania, Lithuania, Bulgaria and Slovenia, all with posterior probabilities of positive excess mortality Lithuania 6,600 (4,500 to 8,700) 3,178 greater than 99%. Excess deaths also reappeared in other coun- Luxembourg 390 (150 to 640) 625 tries that had experienced a medium to large toll in the first wave including Belgium, Spain, Scotland, Northern Ireland, Malta 320 (20 to 640) 304 Sweden, Canada, France and the Netherlands – some at the Montenegro 950 (530 to 1,400) 950 same level (France and Northern Ireland) and others at lower levels (Canada, Scotland, Spain, Belgium, Sweden) than the Netherlands 17,300 (9,400 to 25,300) 15,231 first wave but all lasting for many weeks during this period. New Zealand -1,050 (-3,390 to 1,300) 26 The USA had an even larger increase in mortality compared to the no-pandemic baseline after mid-September than it had in Northern Ireland 2,300 (1,300 to 3,300) 2,751 the first wave and summer months, making it the only coun- Norway 490 (-1,090 to 2,100) 608 try to maintain a steady burden of excess mortality. There were nonetheless variations in excess deaths over time across Poland 82,300 (62,500 to 101,400) 42,171 different states in the USA (Figure 8), reflecting the autonomy Page 10 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Malta Northern Ireland CyprusMontenegro Estonia (140)(320) Serbia Croatia (2,300) (950) (790) Luxembourg (390) Bulgaria (10,300) (6,400) Denmark Latvia Finland (18,200) (2,400) (2,200) (1,200) Lithuania Greece Slovenia Slovakia (6,600)France (3,700) (3,600) (62,700) (11,300) Austria Scotland Chile (9,100) (7,100) (18,400) Hungary (14,300) Switzerland Sweden (10,200) (9,900) Canada (19,800) Belgium Netherlands (17,900) (17,300) Germany (64,100) Romania Czechia Portugal (45,600) (24,000) (20,700) USA (623,100) Poland Spain (82,300) (76,100) Italy England & Wales (118,800) (102,100) Figure 1. Number of excess deaths due to the first year of the coronavirus disease 2019 (COVID-19) pandemic by country. The size of each rectangle shows the number of deaths from all causes in excess of what would be expected if there had been no COVID-19 pandemic from mid-February 2020 through mid-February 2021 for each country. There are no segments for Australia, New Zealand, Norway, Iceland and South Korea because we estimated no detectable excess deaths or a potential reduction in mortality compared to the no-pandemic baseline. Colour for each country indicates its geographical region: the Pacific (Australia, New Zealand, South Korea), the Americas (Canada, Chile, the USA), Central and Eastern Europe (Austria, Bulgaria, Croatia, Czechia, Estonia, Hungary, Latvia, Lithuania, Montenegro, Poland, Romania, Serbia, Slovakia, Slovenia), Southwestern Europe (Cyprus, France, Greece, Italy, Malta, Portugal, Spain), Northwestern Europe (Belgium, England and Wales, Germany, Luxembourg, the Netherlands, Northern Ireland, Scotland, Switzerland) and Nordic (Denmark, Finland, Iceland, Norway, Sweden). Nebraska North Dakota Alaska Wyoming Maine (540) Connecticut (1,400) (740) (710) Vermont(620) Alabama Mississippi New Mexico (2,900)(7,200) (4,500) Montana New District of(12,600) (9,200) (1,800) Hampshire Columbia Florida West Virginia (1,300) (1,300)Rhode South (35,900) (3,400) Delaware Colorado Island Dakota (7,400) Washington (2,000) (1,900) (1,900) Indiana (4,700) OregonMassachusetts Utah Idaho (13,300) (3,400)(10,600) (2,700) (2,200) Kentucky (8,100) Nevada Iowa Kansas (5,500) (5,200) (5,200) Tennessee (14,200) Maryland (10,600) Oklahoma Arkansas Wisconsin Minnesota Texas (8,400) (6,700) (6,600) (5,800) (57,400) North Carolina Missouri South Carolina Louisiana Virginia (16,500) (12,200) (12,000) (11,600) (11,500) Ohio Georgia Arizona Michigan (22,600) (21,400) (19,000) (18,200) California (71,800) Pennsylvania New York New Jersey Illinois (28,100) (24,600) (24,400) (23,700) Figure 2. Number of excess deaths due to the first year of the coronavirus disease 2019 (COVID-19) pandemic by US state. The size of each rectangle shows the number of deaths from all causes in excess of what would be expected if there had been no COVID-19 pandemic from mid-February 2020 through mid-February 2021 for each state and the District of Columbia. There is no segment for Hawaii because we estimated no detectable excess deaths. The colour of each state indicates its geographical region: Southeast (Alabama, Florida, Georgia, North Carolina, South Carolina, Virginia); Northwest (Alaska, Idaho, Oregon, Washington); Southwest (Arizona, Colorado, New Mexico, Utah); South (Arkansas, Kansas, Louisiana, Mississippi, Oklahoma, Texas); West (California, Nevada); Northeast (Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont); Central (Illinois, Indiana, Kentucky, Missouri, Ohio, Tennessee, West Virginia); East North Central (Iowa, Michigan, Minnesota, Wisconsin); and West North Central (Montana, Nebraska, North Dakota, South Dakota, Wyoming). Page 11 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 A USA USA Czechia Czechia Slovakia Slovakia Poland Poland England & Wales England & Wales Italy Italy Portugal Portugal Spain Spain Romania Romania Slovenia Slovenia Lithuania Lithuania Bulgaria Bulgaria Chile Chile Belgium Belgium Switzerland Switzerland Northern Ireland Northern Ireland Montenegro Montenegro Croatia Croatia Scotland Scotland Sweden Sweden Netherlands Netherlands Hungary Hungary Austria Austria Serbia Serbia France France Luxembourg Luxembourg Malta Malta Latvia Latvia Germany Germany Canada Canada Estonia Estonia Denmark Denmark Greece Greece Cyprus Cyprus Finland Finland Norway Norway South Korea South Korea Australia Australia New Zealand New Zealand Iceland Iceland −20% 0% 20% 40% 1 4 7 10 13 16 19 22 25 28 31 34 37 40 Percent increase in deaths (%) Percent increase in deaths rank B Bulgaria Bulgaria Romania Romania Lithuania Lithuania Czechia Czechia Poland Poland Slovakia Slovakia Portugal Portugal Italy Italy USA USA England & Wales England & Wales Slovenia Slovenia Spain Spain Croatia Croatia Belgium Belgium Montenegro Montenegro Serbia Serbia Hungary Hungary Scotland Scotland Northern Ireland Northern Ireland Switzerland Switzerland Latvia Latvia Austria Austria Netherlands Netherlands Chile Chile Sweden Sweden France France Germany Germany Malta Malta Luxembourg Luxembourg Estonia Estonia Canada Canada Denmark Denmark Greece Greece Finland Finland Cyprus Cyprus Norway Norway South Korea South Korea Australia Australia New Zealand New Zealand Iceland Iceland −200 0 200 400 1 4 7 10 13 16 19 22 25 28 31 34 37 40 Excess deaths per 100,000 population Excess death rate rank Low probability High probability Figure 3. Excess mortality due to the first year of the coronavirus 2019 (COVID-19) pandemic, by country. (A) Posterior distribution of percent increase in deaths from any cause from mid-February 2020 to mid-February 2021. Gold dots show the posterior medians. (B) Posterior distribution of excess deaths from any cause per 100,000 people from mid-February 2020 to mid-February 2021. Gold dots show the posterior medians. In both panels, the right-hand side shows the probability distribution for the country’s rank. Countries are ordered vertically by median increase from smallest (at the bottom) to the largest (at the top). Colour for each country’s name indicates its geographical region: the Pacific (Australia, New Zealand, South Korea), the Americas (Canada, Chile, the USA), Central and Eastern Europe (Austria, Bulgaria, Croatia, Czechia, Estonia, Hungary, Latvia, Lithuania, Montenegro, Poland, Romania, Serbia, Slovakia, Slovenia), Southwestern Europe (Cyprus, France, Greece, Italy, Malta, Portugal, Spain), Northwestern Europe (Belgium, England and Wales, Germany, Luxembourg, the Netherlands, Northern Ireland, Scotland, Switzerland) and Nordic (Denmark, Finland, Iceland, Norway, Sweden). See Figure 10 for results by sex. that states, and their governors and legislatures, had with regard burden across the three periods (Figure 7B). For example, to key responses6,34. the first wave accounted for over half of excess deaths in Scotland, Spain, England and Wales, Canada, Sweden, As a result of these heterogeneous dynamics, there was virtu- Belgium, the Netherlands and Cyprus. At the other extreme, the ally no correlation between excess mortality in the first wave period between mid-September 2020 and mid-February 2021 and the summer period among countries (correlation coefficient accounted for over 90% of excess deaths in Bulgaria, Croatia, of percent increase in the two periods = 0.03), and weakly nega- Czechia, Hungary, Latvia, Montenegro, Poland, Slovakia and tive correlation between excess mortality in the first wave and Slovenia. A similar variation was seen across the US states, with mid-September and later (correlation coefficient = -0.15). This excess deaths along the north-eastern coast (Massachusetts, was translated to a variable distribution of excess mortality New Jersey, Connecticut, New York and District of Columbia) Page 12 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 A B New Jersey Mississippi Arizona New Jersey Mississippi Arizona Texas Alabama California Louisiana New York South Carolina Louisiana Arkansas Georgia Pennsylvania New Mexico New Mexico South Carolina Oklahoma South Dakota South Dakota Alabama Tennessee Connecticut Connecticut District of Columbia Georgia Illinois Missouri Pennsylvania Indiana Oklahoma Texas Delaware Delaware Maryland Ohio Arkansas West Virginia Nevada Rhode Island Rhode Island Illinois Indiana District of Columbia Kansas Michigan North Dakota California Missouri Kentucky Michigan North Dakota Tennessee Nevada Colorado Kansas Montana Maryland Ohio Montana Nebraska Florida Massachusetts Iowa Iowa North Carolina North Carolina Massachusetts Florida Nebraska Wyoming Virginia Kentucky Wyoming Virginia Colorado Idaho New York West Virginia Idaho Utah Wisconsin Alaska Minnesota Minnesota Vermont Wisconsin New Hampshire Vermont Utah New Hampshire Oregon Oregon Alaska Washington Washington Maine Maine Hawaii Hawaii 0% 20% 40% 60% 0 200 400 Percent increase in deaths (%) Excess deaths per 100,000 people Low probability High probability Figure 4. Excess mortality due to the first year of the coronavirus disease 2019 (COVID-19) pandemic, by US state. (A) Posterior distribution of percent increase in deaths from any cause from mid-February 2020 to mid-February 2021. Gold dots show the posterior medians. (B) Posterior distribution of excess deaths from any cause per 100,000 people from mid-February 2020 to mid-February 2021. Gold dots show the posterior medians. States are ordered vertically by median increase from smallest (at the bottom) to the largest (at the top). Colour for each state indicates its geographical region: Southeast (Alabama, Florida, Georgia, North Carolina, South Carolina, Virginia); Northwest (Alaska, Idaho, Oregon, Washington); Southwest (Arizona, Colorado, New Mexico, Utah); South (Arkansas, Kansas, Louisiana, Mississippi, Oklahoma, Texas); West (California, Nevada); Northeast (Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont); Central (Illinois, Indiana, Kentucky, Missouri, Ohio, Tennessee, West Virginia); East North Central (Iowa, Michigan, Minnesota, Wisconsin); and West North Central (Montana, Nebraska, North Dakota, South Dakota, Wyoming). being dominated by the first wave, in some southern states aged 65 years and older. On the other hand, Estonia, Finland (Florida, Arizona, Texas and South Carolina) by the summer, (which had the smallest detectable excess mortality of any coun- and in the northern plains (Wisconsin, North and South Dakota try), USA, Canada, Lithuania and Chile had the largest share and Montana) by the post-September period. of excess deaths in people aged younger than 65 years. Of the 35 countries with a detectable increase in mortality (defined as Age and sex-distribution of excess mortality median estimated increase of >2%) and sufficient data to ana- Countries differed in how excess deaths were distributed across lyse by age group, Canada experienced the largest share of age groups (Figure 9). In Denmark, Sweden, France, Switzerland, excess deaths in those aged younger than 45 years (16% of all Belgium and Slovenia >95% of all excess deaths were in those excess deaths), followed by the USA (5%) and Finland (5%; Page 13 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Australia Austria Belgium Bulgaria Canada Chile Croatia 4,000 2,400 4,000 7,000 1,750 3,200 3,500 4,000 6,500 3,500 1,500 2,000 2,800 3,000 6,000 3,000 3,000 1,250 1,600 2,500 5,500 2,500 1,000 2,400 2,000 2,000 5,000 2,000 1,200 1,500 750 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Cyprus Czechia Denmark England & Wales Estonia Finland France 1,300 400 1,300200 4,000 1,200 20,000 17,500 3,500 1,200350 150 1,1003,000 15,000 1,100 15,000 2,500 1,000 300 1,000 100 12,500 2,000 900 10,000 250 900 10,000 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Germany Greece Hungary Iceland Italy Latvia Lithuania 26,000 3,500 24,000 24,000 4,000 70 800 22,000 3,000 3,500 60 20,000 1,250 700 20,000 3,000 50 16,000 600 1,000 18,000 2,500 16,000 2,500 40 2,000 12,000 500 750 14,000 2,000 30 400 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Luxembourg Malta Montenegro Netherlands New Zealand Northern Ireland Norway 240 5,000 120 120 200 800 500 900 100 100 4,000 160 700 400 800 80 80 300 60 120 3,000 600 700 60 40 500 20080 600 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Poland Portugal Romania Scotland Serbia Slovakia Slovenia 16,000 5,000 2,000 4,000 800 8,000 2,000 14,000 1,7504,000 7,000 3,000 1,750 700 12,000 1,500 1,500 6006,000 10,000 3,000 1,250 5,000 2,000 1,250 500 8,000 2,000 4,000 1,000 1,000 400 6,000 750 300 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 South Korea Spain Sweden Switzerland USA 7,000 2,25020,000 6,500 2,400 2,000 80,000 6,000 15,000 2,000 1,750 70,000 5,500 1,500 10,000 60,000 5,000 1,600 1,250 1,000 50,000 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Figure 5. Weekly number of deaths from mid-February 2020 through mid-February 2021. The points show reported deaths. The turquoise shading shows the credible intervals around the median prediction, from 5% (dark) to 95% (light) in 10% increments. Australia Austria Belgium Bulgaria Canada Chile Croatia 150% 80% 60% 30%10% 100% 100% 60% 75% 40% 20% 0% 40% 50% 20% 50% 50% 10% 20% 25% −10% 0% 0% 0% 0% 0% 0% 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Cyprus Czechia Denmark England & Wales Estonia Finland France 120% 30% 150% 40% 80%100% 20.0% 80% 20% 60% 100% 20% 50% 10.0%10% 40% 40% 50% 0% 0.0% 20% 0% 0% 0% −10% 0% −20% −10.0% 0% 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Germany Greece Hungary Iceland Italy Latvia Lithuania 60% 80% 100% 60% 100% 40% 40% 60% 50% 80% 40% 75% 20% 20% 40% 20% 50% 20% 40% 25% 0% 0% 0% 0%0% 0% −20% 0%−50% −20% 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Luxembourg Malta Montenegro Netherlands New Zealand Northern Ireland Norway 100% 30.0% 100% 75% 75% 20.0%100% 20.0% 50% 50% 10.0% 50% 10.0% 50% 50% 25% 0.0% 25% 0.0% 0% 0% −10.0%0% 0% −20.0% 0% −10.0% −25% 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 Poland Portugal Romania Scotland Serbia Slovakia Slovenia 120% 100% 75%80% 100%80% 80% 100% 50% 50% 50% 40% 25% 40% 40% 50% 0% 0% 0% 0% 0% 0% 0%−50% 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 South Korea Spain Sweden Switzerland USA 20.0% 80% 50.0% 10.0% 150% 50% 60% 40.0% 100% 25% 40% 30.0% 0.0% 20% 20.0%50% −10.0% 0% 0% 10.0%0% 0.0% 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 16/02 17/05 16/08 15/11 14/02 <90% probability of increase >90% probability of increase of up to 50% >90% probability of increase of 50%−100% >90% probability of increase of 100% or higher Figure  6. Weekly percent increase in mortality due to the coronavirus disease 2019 (COVID-19) pandemic by country. The turquoise shading shows the credible intervals around the median prediction, from 5% (dark) to 95% (light) in 10% increments. The background shading (grey/yellow/orange/red shading under the turquoise ribbons) indicates the magnitude of the weekly increase that was detectable with a posterior probability of at least 90%. noting that excess death rates in Finland, although detectable, society, such as workers or care home residents, were exposed were lower than in other countries). The high mortality toll in to infection. Percent increase in mortality was similar between younger Canadians may have been due to COVID-19 death at men and women in most countries (Figure 10). There were home35 and an increase in deaths from drug overdose36. This nonetheless some exceptions, e.g. in Chile, Montenegro, Serbia division arises largely from how much specific segments of the and the Netherlands deaths increased by a larger percent Page 14 of 32 Percent increase in deaths (%) Number of weekly deaths Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 A Slovakia Czechia Poland 40% 40% Slovenia Bulgaria Lithuania Montenegro Romania Switzerland USA Croatia Hungary Portugal Chile Serbia Austria Italy 20% Romania Portugal USA 20% Latvia Luxembourg Northern Ireland England & Wales Montenegro Serbia Poland Germany Germany Malta Chile Netherlands Belgium Spain Bulgaria Lithuania Estonia Estonia France South Korea Greece Denmark Canada DenmarkGreece Sweden Scotland Czechia Latvia Finland Cyprus Sweden Italy Spain Finland Canada 0% Croatia Slovenia Austria Northern Ireland Belgium New Zealand 0% Norway Slovakia Norway Switzerland Scotland South Korea Australia Cyprus Hungary Malta France Netherlands England & Wales Iceland Luxembourg Australia Iceland New Zealand −20% −20% −20% 0% 20% 40% −20% 0% 20% 40% Percent increase from mid−February 2020 to the end of May 2020 B 120% 80% 40% 0% Mid−February 2020 to May 2020 June 2020 to mid−September 2020 Mid−September 2020 to mid−February 2021 Figure 7. Excess mortality due to the coronavirus disease 2019 (COVID-19) pandemic in different time periods. (A) Comparison of percent increase in mortality from any cause in excess of what would be expected if there had been no COVID-19 pandemic in summer (beginning of June 2020 to mid-September 2020) and subsequent waves (mid-September 2020 to mid-February 2021) with the first wave (mid-February 2020 to end of May 2020) in each country. (B) Proportion of excess deaths in each of the above three periods in each country. There are no bars for Australia, New Zealand, Norway, Iceland and South Korea in panel B because we estimated no detectable excess deaths or a potential reduction in mortality compared to the no-pandemic baseline. Colour for each country indicates its geographical region: the Pacific (Australia, New Zealand, South Korea), the Americas (Canada, Chile, the USA), Central and Eastern Europe (Austria, Bulgaria, Croatia, Czechia, Estonia, Hungary, Latvia, Lithuania, Montenegro, Poland, Romania, Serbia, Slovakia, Slovenia), Southwestern Europe (Cyprus, France, Greece, Italy, Malta, Portugal, Spain), Northwestern Europe (Belgium, England and Wales, Germany, Luxembourg, the Netherlands, Northern Ireland, Scotland, Switzerland) and Nordic (Denmark, Finland, Iceland, Norway, Sweden). In some countries, there was a reduction in mortality relative to a no-pandemic baseline in some weeks, shown as negative numbers. The country’s total excess death toll is the net effect of these reductions and increases in other periods, with all bars adding to 100%. See Figure 6 for weekly percent increase in mortality. in men (12%–16%) than women (6%–9%); in contrast, in series data from previous years to estimate how many deaths Slovenia, women (15%) experienced a slightly larger percent would be expected in the absence of pandemic through early increase than men (14%). 2021. The models incorporated important features of mortal- ity, including seasonality of death rates, how mortality in one Strengths and limitations week or year may depend on previous week(s) and year(s), and The main strength of our work is the development and appli- the seasonally-variable role of temperature. To our knowl- cation of a method to systematically and consistently use time edge, our models are the only ones that formally incorporated Page 15 of 32 Share of excess Percent increase from June 2020 to mid−September 2020 Cyprus Scotlan E dngla Sn pd a & in Wale S sw N ee dt eh nerland B selgium Ital C yanada Nor Ft rh ae nr cn e Ireland Finla D ndenmark Lu Ux Se Ambourg C Sw hii lt ezerland Eston P iaortu Percent increase from mid−September 2020 to mid−February 2021gal Malta Aust G riaerma L ni ythuania Polan R doman S ialovakia Greec H eungar C yzech S ialovenia Croat B iaulgaria Serbia L M ao tvn iatenegro Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 A 100% 100% 75% 75% South Dakota California 50% 50% Kansas North Dakota Arizona Arizona Montana Oklahoma Texas New Mexico Florida Texas Mississippi NevadaArkansas Nebraska Mississippi Georgia Nevada South Carolina Georgia Wyoming Alabama Missouri Pennsylvania Idaho Arkansas Alabama Louisiana Tennessee Illinois Alaska California Kentucky Idaho Kentucky25% New Mexico 25% Indiana Delaware Alaska Iowa Rhode IslandTennessee Missouri Virginia New YorkNorth Carolina West VNirogritnhi aCarolina Colorado Illinois District of Columbia LouisianaWyoming OklahomaIowa Colorado Minnesota Connecticut New Jersey Maryland Utah Oregon Ohio MarylandUtah Kansas Indiana District of ColumbiaOhio South Dakota Delaware New York Wisconsin New HampshirSeouth CarolinaMichigan New HampshRirehode Island Maine Florida Vermont Virginia 0% Montana Minnesota Vermont Connecticut New Jersey Massachusetts 0% Oregon West Virginia Michigan Hawaii Washington Nebraska Massachusetts Maine Washington Pennsylvania Hawaii Wisconsin North Dakota 0% 25% 50% 75% 100% 0% 25% 50% 75% 100% Percent increase from mid−February 2020 to the end of May 2020 B 80% 40% 0% Mid−February 2020 to May 2020 June 2020 to mid−September 2020 Mid−September 2020 to mid−February 2021 Figure 8. Excess mortality due to the coronavirus disease 2019 (COVID-19) pandemic in different time periods for US States.  (A) Comparison of percent increase in mortality from any cause in excess of what would be expected if there had been no COVID-19 pandemic in summer (beginning of June 2020 to mid-September 2020) and subsequent waves (mid-September 2020 to mid-February 2021) with the first wave (mid-February 2020 to end of May 2020) in each state. (B) Proportion of excess deaths in each of the above three periods in each state. There is no bar for Hawaii because we estimated no detectable excess deaths. In some states, there was a reduction in mortality relative to a no-pandemic baseline in some weeks, shown as negative numbers. The state’s total excess death toll is the net effect of these reductions and increases in other periods, with all bars adding to 100%. Colour for each state indicates its geographical region: Southeast (Alabama, Florida, Georgia, North Carolina, South Carolina, Virginia); Northwest (Alaska, Idaho, Oregon, Washington); Southwest (Arizona, Colorado, New Mexico, Utah); South (Arkansas, Kansas, Louisiana, Mississippi, Oklahoma, Texas); West (California, Nevada); Northeast (Connecticut, Delaware, District of Columbia, Maine, Maryland, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont); Central (Illinois, Indiana, Kentucky, Missouri, Ohio, Tennessee, West Virginia); East North Central (Iowa, Michigan, Minnesota, Wisconsin); and West North Central (Montana, Nebraska, North Dakota, South Dakota, Wyoming). the role of temperature on weekly mortality, and accounted for model choice. As a result, our approach gives a more com- dependency of mortality in one week on preceding week(s) and plete picture of the inherent uncertainty in how many excess in one year on preceding year(s). This methodology allows deaths the pandemic has caused than approaches that are not more robust estimation of the total impacts of the pandemic, probabilistic or use a single model. especially as more time elapses since the beginning of the pan- demic. It also enables comparisons of excess deaths across A limitation of our work is that we did not have data on under- countries on a real-time basis. By modelling death rates, rather lying cause of death. Having a breakdown of deaths by under- than simply the number of deaths as is done in most other anal- lying cause will help develop cause-specific models and yses, we account for changes in population size and age struc- understand which causes have exceeded or fallen below the ture. We used an ensemble of models which typically leads levels expected. Nor did we have data on total mortality by to more robust projections and better accounts for both the individual or community sociodemographic status to understand uncertainty associated with each individual model and that of inequalities in the impacts of the pandemic beyond deaths Page 16 of 32 Percent increase from June 2020 to mid−September 2020 Ma Share of excessssac N hue sw e C J tt e son rn sD ei e ystr ci N tc e ict o w u f C t Yo R oh lo u rk d me b I ias M lai nch d M iga ar ny D lae nla dwa I rl eli V ne or is L mo oP u ne is tN ne n i w s ana yH lva am np ia C sho il ro era In do M di in an ne asot W Via r a g s ih ni in ag G toe nM ois rs gi is as M ipis ps iour O i N ho ir oth IC owa a W ro i ls inc aons Percent increase from mid−September 2020 to mid−February 2021So Ala iu nth b aC m N ae r a w o liM nae K xe icn otuc W O kr yes et gV oir n N g N e ini o br r a th a sD ka ako F tl a C ora idlif aor A nr iaizo K na T ae nn sn ae ssse O T e e k xla ah so N me ava S do auth U tD aa h A kr ok ta ansa I sd M ao hn o W ta y no ami A nglask M aaine Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 100% 75% 50% 25% 0% rk en ce m nda u ni a s ny ly ld a y s sain ga tia ce hia via rg kia ria r u nd le nd nia o d a a au t a r gr n bi ri ni SA il e nia nd da nm we d an lgi rla ve n a t r I pe e o rla rm S rt u oa ee ec a t o va us g yp ola a ela a e tla er a o h r r L b n a l a z o W r m n o S t U ulg s C hu in an a e S F B itz SlD w h e e Po C G C l A u P I e et G xe m S H C & n Ro nt Sc B E Lit F C S N Lu d er o gla n rth M En N o 0−44 45−64 0−64 65+ Figure  9. Distribution of excess deaths due to the first year of the coronavirus disease 2019 (COVID-19) pandemic by age group. The figure shows the share of excess deaths in each age group by country. There are no bars for Australia, New Zealand, Norway, Iceland and South Korea because we estimated no detectable excess deaths or a potential reduction in mortality compared to the no- pandemic baseline. There is no bar for Malta because we only made all-age estimates for reasons described in Methods. For Luxembourg, Cyprus, Latvia, Northern Ireland, Montenegro and Estonia, analysis was done for 0–64 years without a further split into 0–44 years and 45–64 years for reasons described in Methods. Colour for each country indicates its geographical region: the Pacific (Australia, New Zealand, South Korea), the Americas (Canada, Chile, the USA), Central and Eastern Europe (Austria, Bulgaria, Croatia, Czechia, Estonia, Hungary, Latvia, Lithuania, Montenegro, Poland, Romania, Serbia, Slovakia, Slovenia), Southwestern Europe (Cyprus, France, Greece, Italy, Malta, Portugal, Spain), Northwestern Europe (Belgium, England and Wales, Germany, Luxembourg, the Netherlands, Northern Ireland, Scotland, Switzerland) and Nordic (Denmark, Finland, Iceland, Norway, Sweden). assigned to COVID-19 as the underlying cause of death. Where while we estimated no detectable change in mortality. Nonethe- data have been analysed for population subgroups, excess less, the 95% credible intervals of our estimates contained those mortality tends to be higher in marginalised individuals and of Financial Times and The Economist. communities37–39. More detailed data will allow more granu- lar analysis of the impacts of the pandemic, which can in turn The Institute for Health Metrics and Evaluation has released inform resource allocation and a more targeted approach to numbers of excess deaths by fitting a model for seasonality (the mitigating both the direct and indirect effects of the COVID-19 details of the seasonal model are not currently available) and pandemic. projecting the residuals for pre-2020 using a spline model. The models do not account for temperature, as ours do, but hot sum- Comparison with other estimates mer weeks with particularly large deaths were excluded. Sev- The Financial Times and The Economist’s excess deaths tracker eral sources have commented that the estimates are likely an report the number of excess deaths for various countries based overestimate, especially in their earlier version41–43. For exam- on comparisons of deaths in 2020 and 2021 with 2015–2019 ple, the Institute estimated ~156,800 excess deaths for the averages. This approach does not account for general trends UK for the same period as our analysis, compared to ~111,500 in mortality nor for factors like temperature that affect mortal- by us and ~118,500 by the national statistical offices for ity and vary from year to year. The Economist has also recently England, Wales, Scotland and Northern Ireland. They estimated published a set of excess deaths estimates using data from the ~573,500 excess deaths for the USA (revised downwards from Human Mortality Database and the World Mortality Dataset40, an estimate of ~760,000 in September 2021) compared to and an ensemble of gradient boosted decision trees. Countries ~623,100 by us after accounting for temperature. They esti- with small, medium and large number of excess deaths are largely mated ~33,900 deaths for Canada, compared to ~19,800 by us consistent between our analysis and these sources. There are and ~20,900 by Statistics Canada, and ~31,100 excess deaths nonetheless some differences. For example, we estimated for Portugal, compared to ~20,700 by us. EuroMoMo fits a ~76,100 excess deaths for Spain, compared to ~81,300 by sinusoidal seasonal model to death counts but does not report Financial Times and ~86,300 and ~89,600 respectively by the two country-specific excess deaths and hence could not be compared models from The Economist. Our median excess death estimate with our results. for Denmark was about twice as large as that of Financial Times, and those for Greece and Serbia about one third smaller. Simi- The UK Office for National Statistics (ONS) calculated a larly, The Economist’s two models predicted a mortality deficit number of age-standardised measures of excess mortality for 15 of ~2,900 and ~3,200 deaths, respectively, for South Korea, European countries based on comparisons of deaths in 2020 with Page 17 of 32 Share of excess Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Women Men Slovakia Czechia Czechia Poland Portugal Slovakia Poland Italy Slovenia England & Wales England & Wales Bulgaria Spain Romania Italy Chile Romania Lithuania Belgium Spain Lithuania Portugal Bulgaria Montenegro Chile Switzerland Switzerland Belgium Croatia Slovenia Scotland Serbia Hungary Netherlands Montenegro Sweden France Scotland Sweden Croatia Austria Austria Netherlands Hungary Latvia France Luxembourg Luxembourg Germany Latvia Serbia Canada Canada Germany Denmark Estonia Estonia Denmark Greece Greece Finland Finland Norway Norway Cyprus Cyprus Australia Australia New Zealand New Zealand −20% 0% 20% 40% −20% 0% 20% 40% Percent increase in deaths (%) Percent increase in deaths (%) Bulgaria Bulgaria Lithuania Romania Romania Lithuania Czechia Poland Portugal Czechia Slovakia Slovakia Italy Italy Slovenia Serbia Poland Portugal England & Wales Montenegro Belgium England & Wales Croatia Spain Spain Croatia Hungary Slovenia Scotland Belgium Latvia Hungary Montenegro Scotland Switzerland Switzerland Serbia Netherlands Austria Latvia France Chile Sweden Austria Chile Sweden Netherlands France Germany Germany Luxembourg Estonia Estonia Luxembourg Canada Canada Denmark Denmark Greece Greece Finland Finland Norway Norway Cyprus Cyprus Australia Australia New Zealand New Zealand −200 0 200 400 −200 0 200 400 Excess deaths per 100,000 people Excess deaths per 100,000 people Low probability High probability Figure 10. Excess mortality due to the first year of the coronavirus disease 2019 (COVID-19) pandemic, by country and sex. (A) Posterior distribution of percent increase in deaths from any cause from mid-February 2020 to mid-February 2021. Gold dots show the posterior medians. (B) Posterior distribution of excess deaths from any cause per 100,000 people from mid-February 2020 to mid- February 2021. Gold dots show the posterior medians. Countries are ordered vertically by median increase from smallest (at the bottom) to the largest (at the top). Data for Northern Ireland, South Korea and USA were only available for both sexes combined and did not allow sex-specific results. There are no segments for Malta and Iceland because estimates for these countries were only made for both sexes combined, for reasons described in Methods. Colour for each country indicates its geographical region: the Pacific (Australia, New Zealand, South Korea), the Americas (Canada, Chile, the USA), Central and Eastern Europe (Austria, Bulgaria, Croatia, Czechia, Estonia, Hungary, Latvia, Lithuania, Montenegro, Poland, Romania, Serbia, Slovakia, Slovenia), Southwestern Europe (Cyprus, France, Greece, Italy, Malta, Portugal, Spain), Northwestern Europe (Belgium, England and Wales, Germany, Luxembourg, the Netherlands, Northern Ireland, Scotland, Switzerland) and Nordic (Denmark, Finland, Iceland, Norway, Sweden). 2015-2019 averages44, as did Eurostat for the monthly number included the pre-pandemic months and did not account for inter- of deaths. These analyses did not account for temperature annual variations in temperature. For example, in the northern and holidays, and the Eurostat analysis did not account for hemisphere, the first and last three months of 2020 were on aver- changes in population. The ONS concluded that Norway, age warmer than the average of the past five years but weeks Finland, Denmark and Latvia, Cyprus and Estonia had a mortal- 13-40 were on average slightly cooler. ity deficit whereas our estimates indicated no detectable excess mortality for Norway, and increases from 2 to 8% for the other Discussion countries. Differences between our results and those of the The magnitude of excess mortality in the first wave of the ONS may be partly related to the fact that ONS analysis also COVID-19 pandemic was related to two factors, as seen in Page 18 of 32 B A Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 quantitative and qualitative studies on the response to, and lockdowns), increased their testing capacity to various extents, impacts of, the pandemic’s early waves1,6,8,45–61. First, how well and restarted routine healthcare. There were also improvements countries, and subnational entities such as US states, man- in treatments and protocols following large-scale trials and aged the early months of the pandemic – specifically the agility analyses of routine care data73–75. These changes meant that, of imposing timely lockdowns and other social distancing despite the repeated rise in infections, the mortality toll from measures and border controls (e.g., complete or partial travel COVID-19 and other diseases was lower than the first wave but restrictions and/or quarantine for travellers) and adequate and nonetheless considerable in these countries73. The continued effective testing, contact tracing and isolation of infected indi- death toll in these countries may have been because distanc- viduals and their contacts. Second, how prepared and resil- ing measures were not as stringent as those in the first wave, ient the health and social care system was to control the spread and because testing, contact tracing and isolation support did of infection, in the community as well as in health facilities and not reach the coverage or depth needed to contain transmission, care homes, while continuing routine care. as did those in Iceland and South Korea51,76. This was com- pounded by more transmittable variants and that the second Countries eased or maintained travel restrictions and distanc- wave occurred in winter when more time is spent indoors with ing measures of the first wave to different extents and at different less ventilation. The experience of the USA did not resemble paces5,62. They also differed in terms of testing for surveillance that of any of the other countries. Rather, different states saw a and identifying infected individuals, how well and how fast rise in infections and deaths at different times77, because there they traced contacts, and how they supported the isolation of was little coordinated national response and because peri- infected individuals and their contacts51,59. Australia and New ods of extensive travel, such as Thanksgiving and Christmas Zealand took advantage of their island geographies and pursued holidays, led to spread of infection across states. an approach of disease elimination63 – following strict lockdowns they imposed tight border control which kept cases to sporadic The observed patterns of excess mortality in the first year of small numbers and allowed careful contact tracing and isolation. the pandemic indicate that the pandemic’s death toll in the Iceland, Norway and South Korea did not close their borders but next year is likely to depend on three factors: The first, and put in place various forms and durations of quarantine/isolation most important factor in the countries analysed here will be the and testing for travellers. They also effectively integrated their breadth and pace of vaccination, including whether vaccination well-coordinated public health capabilities64 with modern is extended to school-aged children and the use of boost- biomedical (e.g., genomics) and digital technologies (e.g., ers to enhance immunity especially against new variants of data from credit card transactions, mobile phones and CCTV SARS-CoV-2, because vaccines have been shown to be highly [closed-circuit television] footage), and did widespread symp- effective in preventing (severe) COVID-19 and deaths in tri- tomatic and asymptomatic testing to identify, track and iso- als and in real-world settings78–81. Even with high vaccine late infected individuals and their contacts, and to successfully coverage, some adherence to other measures may be needed suppress the epidemic47,65–69, with additional restrictions only when the number of infections rises, because vaccine efficacy when there was a surge in infections. All three countries also is less than 100%, especially against new variants of SARS- have a strong healthcare system that continued to provide routine CoV-281, and because the morbidity and longer-term health care alongside care for COVID-19 patients. morbidity impacts of infection may be significant. Second, as the direct impacts of the COVID-19 pandemic are reduced At the other extreme, many countries in Central and Eastern through vaccination, the indirect impacts will become more Europe, which had put strict measures in place and had expe- visible. These include how much the backlog of routine care rienced no detectable excess mortality during the first half of and persistently high health system pressure impacts deaths 2020, removed restrictions on travel and social contact in sum- from other conditions, and the impacts on jobs and income. mer of 2020, at times to a greater extent or at a faster pace than Mitigating these requires economic and social policies that gen- their Western European counterparts58,62,70,71. With virtually the erate secure employment and income support, and strengthening entire population still susceptible to infection, this set into health and social care. A third, and perhaps more uncertain fac- motion community transmission, which coincided with the intro- tor, is the magnitude of direct COVID-19 deaths that might be duction of more transmittable variants of SARS-CoV-2 which expected in (northern hemisphere) winter 2021-2022, because were not controlled as fast and as strictly as earlier in 2020, retraction of non-pharmaceutical interventions that mandate leading to their true ‘first wave’ in Autumn 2020 which was or facilitate social distancing and use of masks before the entire equivalent to or worse than those in their Western European population is vaccinated may lead to circulating SARS-CoV-2 counterparts in magnitude and duration (Figure 6 and Figure 7). infections in countries as a whole as well as in specific Some Mediterranean countries, such as Malta and Greece, geographical and sociodemographic subgroups of the popula- and Northwestern European countries, such as Austria and tion. In mid-February 2021, vaccination rates were still low Germany, were also largely spared during the first half of in the countries included in our analysis, with the highest 2020, only to see an increase in deaths in autumn and winter, rates in the UK (22% of adults with one dose and 1% with due to a combination of (tourism-related) travel and increased two doses), Serbia (12% and 3%, respectively), the USA local mobility and social interactions72. (11% and 4%, respectively) and Chile (11% and 0.3%, respec- tively). Since then, vaccination accelerated in industrialised Between these extremes, other countries in Europe and Canada countries and emerging economies and in many countries 70% mandated or encouraged masks and face coverings, contin- or more of the population have been vaccinated. Even in ued some forms of distancing measures (including occasional those, specific geographical or social subgroups of the Page 19 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 population may have lower vaccination rates. Further, for coun- England and Wales: https://www.ons.gov.uk/peoplepopulationand- tries where supply and access limit the pace of vaccination, the community/birthsdeathsandmarriages/deaths/datasets/weeklypro- coming year could look as it did for the countries in this paper: visionalfiguresondeathsregisteredinenglandandwales and a choice between lockdowns and a large death toll. To avoid these scenarios, which both have adverse health and well- https://www.ons.gov.uk/peoplepopulationandcommunity/popu- being impacts, vaccine access and roll out must be acceler- lationandmigration/populationestimates/datasets/populationesti- ated and accompanied with actions to both delay and contain matesforukenglandandwalesscotlandandnorthernireland infections, especially new variants of concern, through effec- tive and timely testing, contact tracing and isolation support Germany: https://www.destatis.de/DE/Themen/Gesellschaft- and measures such as mask wearing in crowded indoor settings. Umwelt/Bevoelkerung/Sterbefaelle-Lebenserwartung/Tabellen/ sonderauswertung-sterbefaelle.html Data availability Underlying data New Zealand: https://www.stats.govt.nz/experimental/covid-19- Input data on deaths, population and temperature are available data-portal (time series for category “Total deaths (all causes)” at https://doi.org/10.5281/zenodo.553582912. and indicator “Weekly deaths by age and sex”) This repository contains the following underlying data: Northern Ireland: https://www.nisra.gov.uk/publications/histori- cal-weekly-deaths-data (Historical Weekly Deaths, 2011–2020) • d ata/data.csv (data on deaths, temperature and population and https://www.nisra.gov.uk/publications/weekly-death-statistics- by age group, sex, country and week) northern-ireland-2021 (Weekly Deaths Tables – Week ending 3 Sep- tember 2021) • o utput/result_summaries.csv (weekly estimates of pre- dicted deaths, excess deaths, excess death rates per Scotland: https://www.nrscotland.gov.uk/statistics-and-data/ 100,000 and relative increase in deaths) statistics/statistics-by-theme/vital-events/general-publications/ weekly-and-monthly-data-on-births-and-deaths/deaths-involving- Data are available under the terms of the Creative Commons coronavirus-covid-19-in-scotland/related-statistics (Weekly deaths Zero “No rights reserved” data waiver (CC0 1.0 Public domain by location of death, age group, sex and cause, 2020 and 2021 dedication). and Weekly deaths by sex and age group, 2000 to 2019) The original data sets used in the study are publicly available South Korea: https://kosis.kr/covid_eng/statistics_excessdeath.do from the following locations: and https://mdis.kostat.go.kr/index.do Data on deaths and population United States: https://data.cdc.gov/NCHS/Weekly-counts-of- deaths-by-jurisdiction-and-age-gr/y5bj-9g5w and https://www.cdc. UN World Population prospects: https://population.un.org/ gov/nchs/nvss/bridged_race/data_documentation.htm wpp/Download/Files/1_Indicators%20(Standard)/CSV_FILES/ WPP2019_PopulationByAgeSex_Medium.csv Data on temperature and gridded population Australia: https://www.abs.gov.au/statistics/health/causes-death/ https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/ provisional-mortality-statistics/latest-release (Provisional Mortal- era5 and ity Statistics, Weekly Dashboard, Jan 2020-May 2021.xlsx and Doc- tor certified deaths by week of occurrence, 2015-19.xlsx) https://sedac.ciesin.columbia.edu/data/collection/gpw-v4 Eurostat: https://ec.europa.eu/eurostat/data/database (tables Extended data demo_r_mwk_05 and demo_pjangroup) The computer code for the Bayesian model ensemble used in this study is available at: https://github.com/vkontis/excess_mortality/ Canada: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid= tree/pub2 1310076801 and https://www150.statcan.gc.ca/t1/tbl1/en/tv. action?pid=1710000501 Archived analysis code at time of publication: https://doi. org/10.5281/zenodo.553582912. Chile: https://deis.minsal.cl/#datosabiertos (DEFUNCIONES_ FUENTE_DEIS_2016_2021_23092021.zip) License: GNU Affero General Public License v3.0 Page 20 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 References 1. Kontis V, Bennett JE, Rashid T, et al.: Magnitude, demographics and dynamics  Change. 2014; 4(4): 269–73. of the effect of the first wave of the COVID-19 pandemic on all-cause Publisher Full Text  mortality in 21 industrialized countries. Nat Med. 2020; 26(12): 1919–28. 25. Parks RM, Bennett JE, Tamura-Wicks H, et al.: Anomalously warm  PubMed Abstract | Publisher Full Text  temperatures are associated with increased injury deaths. Nat Med. 2020; 2. Beaney T, Clarke JM, Jain V, et al.: Excess mortality: the gold standard in  26(1): 65–70. measuring the impact of COVID-19 worldwide? J R Soc Med. 2020; 113(9): PubMed Abstract | Publisher Full Text | Free Full Text  329–34. 26. Sørbye SH, Rue H: Penalised Complexity Priors for Stationary  PubMed Abstract | Publisher Full Text | Free Full Text  Autoregressive Processes. J Time Ser Anal. 2017; 38(6): 923–35. 3. Douglas M, Katikireddi SV, Taulbut M, et al.: Mitigating the wider health  Publisher Full Text  effects of covid-19 pandemic response. BMJ. 2020; 369: m1557. 27. Rue H, Martino S, Chopin N: Approximate Bayesian inference for latent  PubMed Abstract | Publisher Full Text | Free Full Text  Gaussian models by using integrated nested Laplace approximations. (with 4. The Academy of Medical Sciences: Coronavirus: preparing for challenges this  discussion) J R Stat Soc Series B. 2009; 71(2): 319–92. winter. 2020. Publisher Full Text  Reference Source 28. Timmermann A: Chapter 4 Forecast combinations. In: Elliott G, Granger C, 5. Han E, Tan MMJ, Turk E, et al.: Lessons learnt from easing COVID-19 Timmermann A, eds. Handbook of Economic Forecasting. 2006; 1: 135–96. restrictions: an analysis of countries and regions in Asia Pacific and Publisher Full Text  Europe. Lancet. 2020; 396(10261): 1525–34. 29. Graefe A, Küchenhoff H, Stierle V, et al.: Limitations of Ensemble Bayesian  PubMed Abstract | Publisher Full Text | Free Full Text  Model Averaging for forecasting social science problems. Int J Forecast. 6. Jung AS, Haldane V, Neill R, et al.: National responses to covid-19: drivers,  2015; 31(3): 943–51. complexities, and uncertainties in the first year of the pandemic. BMJ. Publisher Full Text  2021; 375: e068954. 30. Jones N: How coronavirus lockdowns stopped flu in its tracks. Nature. 2020. PubMed Abstract | Publisher Full Text | Free Full Text  PubMed Abstract | Publisher Full Text  7. Tan MMJ, Neill R, Haldane V, et al.: Assessing the role of qualitative factors in  31. Klein A: Australia sees huge decrease in flu cases due to coronavirus pandemic responses. BMJ. 2021; 375: e067512. measures. NewScientist. 2020. PubMed Abstract | Publisher Full Text | Free Full Text  Reference Source 8. Wu S, Neill R, De Foo C, et al.: Aggressive containment, suppression, and  32. Olsen SJ, Azziz-Baumgartner E, Budd AP, et al.: Decreased influenza activity mitigation of covid-19: lessons learnt from eight countries. BMJ. 2021; 375: during the COVID-19 pandemic - United States, Australia, Chile, and South e067508. Africa, 2020. MMWR Morb Mortal Wkly Rep. 2020; 69(37): 1305–9. PubMed Abstract | Publisher Full Text | Free Full Text  PubMed Abstract | Publisher Full Text | Free Full Text  9. Mathieu E, Ritchie H, Ortiz-Ospina E, et al.: A global database of COVID-19 33. Peek K: Flu Has Disappeared Worldwide during the COVID Pandemic. vaccinations. Nat Hum Behav. 2021; 5(7): 947–953. Scientific American. 2021. PubMed Abstract | Publisher Full Text  Reference Source 10. Office for National Statistics: Quarterly mortality report, England: October to 34. Lewis M: The Premonition: A Pandemic Story. Penguin Books, Limited; 2021. December 2019 and year-end review. 2019. Reference Source Reference Source 35. Young C: ‘Significant number’ of young people are dying at home with 11. European Centre for Medium-Range Weather Forecasts: ERA5 Reanalysis. COVID-19. National Post. 2021. Research Data Archive at the National Center for Atmospheric Research, Reference Source Computational and Information Systems Laboratory. 2017. Publisher Full Text  36. Statistics Canada: Provisional death counts and excess mortality, January 2020 to June 2021. 2021. 12. Vkontis: vkontis/excess_mortality: covid_impacts_ 2020-21 (v1.0). Zenodo. Reference Source 2021. http://www.doi.org/10.5281/zenodo.5535829 37. Davies B, Parkes BL, Bennett J, et al.: Community factors and excess mortality in first wave of the COVID-19 pandemic in England. Nat Commun. 2021; 13. Kontis V, Bennett JE, Mathers CD, et al.: Future life expectancy in 35  12(1): 3755. industrialised countries: projections with a Bayesian model ensemble. PubMed Abstract | Publisher Full Text | Free Full Text  Lancet. 2017; 389(10076): 1323–35. PubMed Abstract | Publisher Full Text | Free Full Text  38. Suleman M, Sonthalia S, Webb C, et al.: Unequal pandemic, fairer recovery. The Health Foundation, 2021. 14. Parks RM, Bennett JE, Foreman KJ, et al.: National and regional seasonal  Reference Source dynamics of all-cause and cause-specific mortality in the USA from 1980 to 2016. eLife. 2018; 7: e35500. 39. Stokes AC, Lundberg DJ, Elo IT, et al.: COVID-19 and excess mortality in the PubMed Abstract | Publisher Full Text | Free Full Text  United States: A county-level analysis. PLoS Med. 2021; 18(5): e1003571. PubMed Abstract | Publisher Full Text | Free Full Text  15. Feinstein CA: Seasonality of Deaths in the U.S. by Age and Cause. Demogr Res. 2002; 6(17): 471–88. 40. Karlinsky A, Kobak D: Tracking excess mortality across countries during  Publisher Full Text  the COVID-19 pandemic with the World Mortality Dataset. eLife. 2021; 10: e69336. 16. Fowler T, Southgate RJ, Waite T, et al.: Excess Winter Deaths in Europe: a  PubMed Abstract | Publisher Full Text | Free Full Text  multi-country descriptive analysis. Eur J Public Health. 2015; 25(2): 339–45. PubMed Abstract | Publisher Full Text  41. Gelman A: Doubting the IHME claims about excess deaths by country. Statistical Modeling, Causal Inference, and Social Science; 2021. 17. McKee CM: Deaths in winter: Can Britain learn from Europe? Eur J Epidemiol. Reference Source 1989; 5(2): 178–82. 42. Dyer O: Covid-19: Study claims real global deaths are twice official figures. PubMed Abstract | Publisher Full Text  BMJ. 2021; 373: n1188. 18. Hyndman RJ, Athanasopoulos G: Forecasting: principles and practice. 2nd PubMed Abstract | Publisher Full Text  edition. Melbourne, Australia: OTexts; 2018. 43. There have been 7m-13m excess deaths worldwide during the pandemic. Reference Source The Economist. 2021. 19. Congdon P: Applied Bayesian Modelling. John Wiley & Sons; 2003. Reference Source Reference Source 44. Office of National Statistics (ONS): Comparisons of all-cause mortality  20. Basu R: High ambient temperature and mortality: a review of  between European countries and regions 2020. 2021. epidemiologic studies from 2001 to 2008. Environ Health. 2009; 8: 40. Reference Source PubMed Abstract | Publisher Full Text | Free Full Text  45. Okell LC, Verity R, Watson OJ, et al.: Have deaths from COVID-19 in Europe 21. Basu R, Samet JM: Relation between elevated ambient temperature and  plateaued due to herd immunity? Lancet. 2020; 395(10241): e110–e111. mortality: a review of the epidemiologic evidence. Epidemiol Rev. 2002; 24(2): PubMed Abstract | Publisher Full Text | Free Full Text  190–202. 46. Haldane V, De Foo C, Abdalla SM, et al.: Health systems resilience in  PubMed Abstract | Publisher Full Text  managing the COVID-19 pandemic: lessons from 28 countries. Nat Med. 22. Gasparrini A, Guo Y, Hashizume M, et al.: Mortality risk attributable to high  2021; 27(6): 964–80. and low ambient temperature: a multicountry observational study. Lancet. PubMed Abstract | Publisher Full Text  2015; 386(9991): 369–75. 47. Lewis D: Why many countries failed at COVID contact-tracing — but some PubMed Abstract | Publisher Full Text | Free Full Text  got it right. Nature. 2020; 588(7838): 384–387. 23. Song X, Wang S, Hu Y, et al.: Impact of ambient temperature on morbidity  PubMed Abstract | Publisher Full Text  and mortality: An overview of reviews. Sci Total Environ. 2017; 586: 241–54. 48. Legido-Quigley H, Asgari N, Teo YY, et al.: Are high-performing health  PubMed Abstract | Publisher Full Text  systems resilient against the COVID-19 epidemic? Lancet. 2020; 395(10227): 24. Bennett JE, Blangiardo M, Fecht D, et al.: Vulnerability to the mortality effects 848–50. of warm temperature in the districts of England and Wales. Nat Clim PubMed Abstract | Publisher Full Text | Free Full Text  Page 21 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 49. Lee JK, Bullen C, Ben Amor Y, et al.: Institutional and behaviour-change  587(7835): 536–539. interventions to support COVID-19 public health measures: a review by the PubMed Abstract | Publisher Full Text  Lancet Commission Task Force on public health measures to suppress the  66. Zastrow M: South Korea is reporting intimate details of COVID-19 cases: has pandemic. Int Health. 2021; 13(5): 399–409. it helped? Nature. 2020. PubMed Abstract | Publisher Full Text | Free Full Text  PubMed Abstract | Publisher Full Text  50. Oh J, Lee HY, Khuong QL, et al.: Mobility restrictions were associated with  67. Kim JH, An JAR, Oh SJ, et al.: Emerging COVID-19 success story: South Korea reductions in COVID-19 incidence early in the pandemic: evidence from a learned the lessons of MERS. Our World in Data - Exemplars in Global Health real-time evaluation in 34 countries. Sci Rep. 2021; 11(1): 13717. platform. Our World in Data; 2021. PubMed Abstract | Publisher Full Text | Free Full Text  Reference Source 51. Moon J, Chekar CK, Barberá D, et al.: Optimising ‘Test and Trace’ Systems: 68. You J: Lessons From South Korea’s Covid-19 Policy Response. Am Rev Public Early lessons from a comparative analysis of six countries. SSRN Electronic Adm. 2020; 50(6–7): 801–8. Journal. 2020. Publisher Full Text  Publisher Full Text  69. Lim B, Kyoungseo Hong E, Mou J, et al.: COVID-19 in Korea: Success Based on 52. Violato C, Violato EM, Violato EM: Impact of the stringency of lockdown  Past Failure. Asian Economic Papers. 2021; 20(2): 41–62. measures on covid-19: A theoretical model of a pandemic. PLoS One. 2021; Publisher Full Text  16(10): e0258205. 70. Klimovsky D, Nemec J: The Impacts of the COVID-19 Pandemic and PubMed Abstract | Publisher Full Text | Free Full Text  Responses from Various Policy Actors in the Czech Republic and Slovakia in  53. Hirsch C: Europe’s coronavirus lockdown measures compared. Politico. 2020. 2020: An Introduction to a Special Issue. SciPap. 2021; 29(1): 1255. Reference Source Publisher Full Text  54. Horton R: The COVID-19 Catastrophe: What’s Gone Wrong and How To Stop 71. Coronavirus: Czechs hold ‘farewell party’ for pandemic. BBC News, 2021. It Happening Again. Wiley; 2021. Reference Source Reference Source 72. Lemey P, Ruktanonchai N, Hong SL, et al.: Untangling introductions and 55. Woolhouse M: The Year the World Went Mad: A Scientific Memoir from the persistence in COVID-19 resurgence in Europe. Nature. 2021; 595(7869): Pandemic. Sandstone Press Limited; 2022. 713–717. Reference Source PubMed Abstract | Publisher Full Text | Free Full Text  56. House of Commons Health and Social Care, Science and Technology 73. Ledford H: Why do COVID death rates seem to be falling? Nature. 2020; Committees: Coronavirus: lessons learned to date. House of Commons; 2021. 587(7833): 190–2. Reference Source PubMed Abstract | Publisher Full Text  57. Hale T, Angrist N, Hale AJ, et al.: Government responses and COVID-19 74. Dennis JM, McGovern AP, Vollmer SJ, et al.: Improving Survival of Critical Care  deaths: Global evidence across multiple pandemic waves. PLoS One. 2021; Patients With Coronavirus Disease 2019 in England: A National Cohort  16(7): e0253116. Study, March to June 2020. Crit Care Med. 2021; 49(2): 209–14. PubMed Abstract | Publisher Full Text | Free Full Text  PubMed Abstract | Publisher Full Text | Free Full Text  58. Webb E, Winkelmann J, Scarpetti G, et al.: Lessons learned from the Baltic  75. Chalmers JD, Crichton ML, Goeminne PC, et al.: Management of hospitalised  countries’ response to the first wave of COVID-19. Health Policy. 2021; S0168- adults with coronavirus disease 2019 (COVID-19): a European Respiratory 8510(21)00293-1. Society living guideline. Eur Respir J. 2021; 57(4): 2100048. PubMed Abstract | Publisher Full Text | Free Full Text  PubMed Abstract | Publisher Full Text | Free Full Text  59. Rajan S, Cylus JD, Mckee M: What do countries need to do to implement  76. Chung SC, Marlow S, Tobias N, et al.: Lessons from countries implementing  effective ‘find, test, trace, isolate and support’ systems? J R Soc Med. 2020; find, test, trace, isolation and support policies in the rapid response of the 113(7): 245–250. COVID-19 pandemic: a systematic review. BMJ Open. 2021; 11(7): e047832. PubMed Abstract | Publisher Full Text | Free Full Text  PubMed Abstract | Publisher Full Text | Free Full Text  60. Lewis T: How the U.S. Pandemic Response Went Wrong—and What Went 77. The National Academy for State Health Policy: 2020 COVID-19 State Right—during a Year of COVID. Scientific American. 2021. Restrictions, Re-openings, and Mask Requirements. 2021. Reference Source Reference Source 61. Henley J: ‘A once-in-a-generation event’: lessons from a year of lockdown in 78. Public Health England, Cambridge University MRC Biostatistics Unit: Direct and  Europe. The Guardian. 2021. indirect impact of the vaccination programme on COVID-19 infections and Reference Source mortality. Public Health England; 2021. Reference Source 62. Hale T, Webster S, Petherick A, et al.: Oxford COVID-19 Government Response 79. Thompson MG, Burgess JL, Naleway AL, et al.: Prevention and Attenuation of  Tracker. Blavatnik School of Government, University of Oxford. 2020. Covid-19 with the BNT162b2 and mRNA-1273 Vaccines. N Engl J Med. 2021; Reference Source 385(4): 320–329. 63. Baker M, Kvalsvig A, Verrall AJ, et al.: New Zealand’s elimination strategy for PubMed Abstract | Publisher Full Text | Free Full Text  the COVID-19 pandemic and what is required to make it work. N Z Med J. 80. Jara A, Undurraga EA, González C, et al.: Effectiveness of an Inactivated SARS- 2020; 133(1512): 10–14. CoV-2 Vaccine in Chile. N Engl J Med. 2021; 385(10): 875–884. PubMed Abstract  PubMed Abstract | Publisher Full Text | Free Full Text  64. Yoo KJ, Kwon S, Choi Y, et al.: Systematic assessment of South Korea’s 81. Eyre DW, Taylor D, Purver M, et al.: Effect of Covid-19 Vaccination capabilities to control COVID-19. Health Policy. 2021; 125(5): 568–576. on Transmission of Alpha and Delta Variants. N Engl J Med. 2022; PubMed Abstract | Publisher Full Text | Free Full Text  NEJMoa2116597. 65. Scudellari M: How Iceland hammered COVID with science. Nature. 2020; PubMed Abstract | Publisher Full Text | Free Full Text Page 22 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Open Peer Review Current Peer Review Status: Version 2 Reviewer Report 21 February 2022 https://doi.org/10.21956/wellcomeopenres.19577.r48703 © 2022 Gupta R. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Rajeev Gupta 1 Academic Research Development Unit, Rajasthan University of Health Sciences, Jaipur, Rajasthan, India 2 Eternal Heart Care Centre and Research Institute, Jaipur, India The article is now acceptable for indexing. Competing Interests: I have contributed to data on non-communicable disease risk factors to NCDRISC (collaboration). Some of the authors in the article are members of that group. I have not been a member of the writing group of any of the articles emanating from NCDRISC. Reviewer Expertise: Cardiovascular epidemiology I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Reviewer Report 17 February 2022 https://doi.org/10.21956/wellcomeopenres.19577.r48702 © 2022 Gómez-Rubio V. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Virgilio Gómez-Rubio Department of Mathematics, School of Industrial Engineering-Albacete, Universidad de Castilla-La Mancha, Ciudad Real, Spain Thanks for your reply and the changes in the manuscript that address all my concerns raised in my previous review.   Page 23 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 Competing Interests: No competing interests were disclosed. Reviewer Expertise: Statistics I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. Version 1 Reviewer Report 06 January 2022 https://doi.org/10.21956/wellcomeopenres.19069.r47314 © 2022 Gómez-Rubio V. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Virgilio Gómez-Rubio Department of Mathematics, School of Industrial Engineering-Albacete, Universidad de Castilla-La Mancha, Ciudad Real, Spain The authors have presented an interesting analysis of the excess mortality in 40 countries during the first year of the COVID-19 pandemic. They have used publicly available data and Bayesian hierarchical models for the analysis. They have also provided links to the data sources as well as a link to a GitHub repository with the R code and datasets to make sure that the results can be replicated. As part of the analysis, they conduct a validation of the model which makes me think that the results obtained (and their conclusions) are reliable. The conclusions are supported by the data and the analysis carried out. My only concern is about the "model averaging approach" followed by the authors. First of all, why not select a single model using the different model selection criteria available in INLA? Some of the models are nested so it makes me wonder whether the 'extra' components are needed. Most likely the answer is yes, and then the authors should choose the model with the extra components. Secondly, I agree that following a Bayesian model averaging is useful but I am not sure that this can be achieved by simply "pooling" the samples obtained from INLA (see, for example, Gómez- Rubio et al., 20201 where we illustrate how to do BMA with INLA by weighting the different models according to the marginal likelihood). The authors should have weighted the samples according to the values of the marginal likelihood for each model (which are reported by INLA). These values could have been used to compute the posterior probability of each model for model selection as well. Having said this, it looks like the results produced are similar to those provided by other analyses, which may indicate that weighting the samples have no major impact (i.e., all models produce very similar death estimates), but I think this is something that the authors should check.   Page 24 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 References 1. Gómez-Rubio V, Bivand R, Rue H: Bayesian Model Averaging with the Integrated Nested Laplace Approximation. Econometrics. 2020; 8 (2). Publisher Full Text Is the work clearly and accurately presented and does it cite the current literature? Yes Is the study design appropriate and is the work technically sound? Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes If applicable, is the statistical analysis and its interpretation appropriate? Partly Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Yes Competing Interests: No competing interests were disclosed. Reviewer Expertise: Statistics I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Author Response 01 Feb 2022 Majid Ezzati, Imperial College London, London, UK My only concern is about the "model averaging approach" followed by the authors. First of all, why not select a single model using the different model selection criteria available in INLA? Some of the models are nested so it makes me wonder whether the 'extra' components are needed. Most likely the answer is yes, and then the authors should choose the model with the extra components. This is a good point. We used a model ensemble and model averaging approach for three reasons: First, we have shown in prior work1 that for forecasts and projections, which by definition involve out-of-sample estimation, a model ensemble/average outperforms the best available model when the projection timeline is longer than a few time units (which would be a few weeks here) – see Appendix Figure 3 in Kontis et al.1. This is consistent with findings from other works that involve forecasting and projections2-4. Second, which single   Page 25 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 model performs best in out-of-sample prediction depends on the country and year, even among those that have related terms, as seen in Appendix Figure 1 of Kontis et al..1. This year-to-year and country-to-country variability in what the best model is, is itself due to the complex behaviour of mortality data over space and time, and means that the best model for the pandemic period is uncertain. In our validation analyses, 13 out of 16 models ranked best (i.e., had the smallest absolute projection error) for at least one country and one validation year. Even when looking at individual countries, in only 2 out of 33 countries did the same model emerge as the best model in all three validation years. What model is best is not only about choosing the one with the most components, but also the choice of prior (e.g., the order of the autoregressive term). Finally, and as a consequence of the aforementioned issues, the use of multiple models and model averaging provides a more complete picture of the inherent uncertainty in how many excess deaths the pandemic has caused than approaches that are not probabilistic or use a single model. Secondly, I agree that following a Bayesian model averaging is useful but I am not sure that this can be achieved by simply "pooling" the samples obtained from INLA (see, for example, Gómez-Rubio et al., 2020 where we illustrate how to do BMA with INLA by weighting the different models according to the marginal likelihood). The authors should have weighted the samples according to the values of the marginal likelihood for each model (which are reported by INLA). These values could have been used to compute the posterior probability of each model for model selection as well. Having said this, it looks like the results produced are similar to those provided by other analyses, which may indicate that weighting the samples have no major impact (i.e., all models produce very similar death estimates), but I think this is something that the authors should check. Although in-sample measures of fit are well-suited for applications where there is an interest in the parameters of the model, for projection tasks like ours, models must be evaluated and weighted based on their out-of-sample performance, which is what they have been designed for. Projections of individual models can be combined in a fully Bayesian approach, by assigning weights based on out-of-sample performance, or simply by assigning uniform weights (i.e., using an unweighted average). In this work, we used the latter approach for the following reasons: First, methods for combing projections in a fully Bayesian manner typically use simpler base models (e.g. simple linear regressions)4,5 and in our case, such an approach would not have been feasible due to its computational complexity. Second, calculating weights based on out-of-sample performance requires further validation to decide how weights should be defined (e.g., as the reciprocal of absolute projection error, or in some other way that penalises models with larger errors), as we have done in prior work1. Validating the choice of model weights in turn requires an additional hold-out period which in our case was not available for many countries, due to the short duration of their time series. Choosing uniform model weights avoids this problem and has been shown to produce equal or superior results to using non-equal weights in many applications2,6. Nonetheless, based on this comment, we now include a sensitivity analysis on how the different models are weighted. Specifically, in the sensitivity analysis, the number of draws from each model was inversely proportional to the absolute error of prediction in the   Page 26 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 validation analyses. As seen, the results are robust to this choice, with the current approach having the advantage of being usable for countries with shorter time series. References 1. Kontis V, Bennett JE, Mathers CD, Li G, Foreman K, Ezzati M. Future life expectancy in 35 industrialised countries: projections with a Bayesian model ensemble. Lancet 2017; 389 (10076): 1323-35. 2. Clemen RT. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting 1989; 5(4): 559-83. 3. Makridakis S, Winkler RL. Averages of Forecasts: Some Empirical Results. Management Science 1983; 29(9): 987-96. 4. Raftery AE, Gneiting T, Balabdaoui F, Polakowski M. Using Bayesian Model Averaging to Calibrate Forecast Ensembles. Monthly Weather Review 2005; 133(5): 1155-74. 5. Eklund J, Karlsson S. Forecast Combination and Model Averaging Using Predictive Measures. Econometric Reviews 2007; 26(2-4): 329-63. 6. Timmermann A. Chapter 4 Forecast Combinations. In: Elliott G, Granger CWJ, Timmermann A, eds. Handbook of Economic Forecasting: Elsevier; 2006: 135-96. Competing Interests: No competing interests were disclosed. Reviewer Report 18 November 2021 https://doi.org/10.21956/wellcomeopenres.19069.r46937 © 2021 Gupta R. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Rajeev Gupta 1 Academic Research Development Unit, Rajasthan University of Health Sciences, Jaipur, Rajasthan, India 2 Eternal Heart Care Centre and Research Institute, Jaipur, India General comments: 1. This is an interesting article on the spread of COVID-19 in 40 industrialized countries. However, the article has limited information on the lessons learned from these studies. This is the major weakness of the study. Qualitative studies should be mentioned.   2. Many books have been written on the topic and it would be useful for the authors to look at qualitative information from some of them. I would specifically mention the following (which I have read (sic) and many others that I could not): 1. Michael Lewis. The premonition: a pandemic story.     Page 27 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 2. Niall Ferguson: Doom: the politics of catastrophe.   3. Jeremy Farrar and Anjana Ahuja. Spike: The virus vs the people, the inside story.   4. Adam Kucharaski. The rules of contagion.   5. Many journal as well as newspaper articles also provide insights into the failures and successes.   3. Would it be worthwhile to add the delta wave epidemic that swept through India, UK and US and is currently creating havoc in Europe? Specific comments: Abstract: 1. Data from US and US states have been prominently displayed in the Results section but are missing.   2.  The conclusion statement does not mention the lessons missed (NPI, delays, etc). Methods: 1. Reference to the “Our World in Data” website is not provided.   2. The authors have extensively adjusted for the temperature variation data. This is important as this is an important determinant. Two other variables could be important: (a) Age distribution of populations in these countries; and (b) Ambient pollution. Data of both are widely available and it would be interesting to evaluate whether these variables influence the outcomes.   3. Page 7, para 2, line 2: is it mid-February 2021? Please correct. Results: 1. Do we have data on excess mortality for US states? May be interesting to add (unless it is part of a forthcoming study).   2. No data are presented regarding “lessons learned”. May be useful to study the social determinants such as GDPs, income, and health systems. Discussion: 1. Please discuss the data presented and their limitations, and then focus on the implications of the study. Is the work clearly and accurately presented and does it cite the current literature? Partly Is the study design appropriate and is the work technically sound? Yes Are sufficient details of methods and analysis provided to allow replication by others? Yes If applicable, is the statistical analysis and its interpretation appropriate?   Page 28 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 I cannot comment. A qualified statistician is required. Are all the source data underlying the results available to ensure full reproducibility? Yes Are the conclusions drawn adequately supported by the results? Yes Competing Interests: I have contributed to data on non-communicable disease risk factors to NCDRISC (collaboration). Some of the authors in the article are members of that group. I have not been a member of the writing group of any of the articles emanating from NCDRISC. Reviewer Expertise: Cardiovascular epidemiology I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above. Author Response 01 Feb 2022 Majid Ezzati, Imperial College London, London, UK 1. This is an interesting article on the spread of COVID-19 in 40 industrialized countries. However, the article has limited information on the lessons learned from these studies. This is the major weakness of the study. Qualitative studies should be mentioned. The revised paper refers to over 25 qualitative and evidence synthesis studies, either across countries or for specific countries. 2. Many books have been written on the topic and it would be useful for the authors to look at qualitative information from some of them. I would specifically mention the following (which I have read (sic) and many others that I could not):   1. Michael Lewis. The premonition: a pandemic story. 2. Niall Ferguson: Doom: the politics of catastrophe. 3. Jeremy Farrar and Anjana Ahuja. Spike: The virus vs the people, the inside story. 4. Adam Kucharaski. The rules of contagion. 5. Many journal as well as newspaper articles also provide insights into the failures and successes. We are aware of the large number of books, and the even larger number of newspaper articles and opinion pieces, on the COVID-19 pandemic. We have cited some key ones that are directly related to our paper including books as well as newspaper articles. 3. Would it be worthwhile to add the delta wave epidemic that swept through India, UK and US and is currently creating havoc in Europe?   Page 29 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 We believe that the Delta and Omicron waves should be the subject of distinct analyses for two reasons: First, as stated in the Introduction of our paper, the effectiveness of vaccination in reducing deaths motivates distinct analyses and interpretation prior to v versus after mass vaccination, with the focus of our paper on the former. Second, because the number of deaths that would be expected had the pandemic not occurred is not directly measurable, its estimation requires projection modelling based on past time-series of weekly deaths. The longer the period of projection since the beginning of the pandemic, the more difficult and uncertain the projections, possibly requiring methods that are different from those used for the first year. Specific comments: Abstract: Data from US and US states have been prominently displayed in the Results section but are missing. We have included in the abstract as correctly suggested. The conclusion statement does not mention the lessons missed (NPI, delays, etc). We have made the use of NPIs explicit in the abstract’s conclusion, from its original implicit use. Methods: Reference to the “Our World in Data” website is not provided. We had initially done this as a hyperlink. We have now also cited the corresponding publication. The authors have extensively adjusted for the temperature variation data. This is important as this is an important determinant. Two other variables could be important: (a) Age distribution of populations in these countries; and (b) Ambient pollution. Data of both are widely available and it would be interesting to evaluate whether these variables influence the outcomes. Our statistical models were designed to estimate weekly death rates, had the pandemic not occurred. With this objective, the models do not adjust for temperature or any other variable. Rather the models include, as covariates, variables that are predictors of mortality, vary from week to week (because those that do not vary from week to week are captured by other model terms), and are themselves not affected by the pandemic (so that we make unbiased estimates of counterfactual, no-pandemic mortality). Temperature was included in our model because it is an established predictor of short-term mortality, and was itself not affected by the pandemic. We did not include air pollution in the model, because the pandemic and the associated policy and behavioural responses, in   Page 30 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 particular the reduction in vehicular traffic and industrial activity, led to changes in air pollution1,2. This in turn means that the observed pollution is different from what pollution levels would have been without the pandemic. In contrast, researchers have estimated a short-term pandemic impact of only ~0.01°C on ambient temperature3, which is orders of magnitude smaller than seasonal and interannual variation. The age distribution of each country is already embedded in past data on deaths and population, and is therefore implicitly used to produce our projections. More details provided below. As the Reviewer correctly points out, our main results are based on analysis that used data from all ages together, for three reasons: First, at least three terms in our model implicitly take into account any change in age distribution that might occur from week to week, which is the relevant time interval for our work: (1) The model has a random intercept for each week of the year (θweek) which allows deaths to change from week to week, some of which may be partly due to differential changes in age groups; the weekly terms repeat across years although their magnitude can change due to variations in temperature. (2) The autoregressive structure of the model allows these terms to be related to deaths in the previous week which means that phenomena that gradually change the age structure of deaths on a weekly scale (e.g., weekly dynamics of flu) are captured. (3) Similarly, we have a term that captures monthly mortality, also with autoregressive structure at annual scale, which implicitly captures phenomena that change deaths at annual scale in a correlated way. Second, when the time unit of analysis is the week, rather than the year as is the case in most mortality projections, the number of deaths in many country-week-age units is zero or small. Using all ages together can make model fits more stable, hence trading off age- related precision with how well model parameters are estimated. Third, running a single model for all ages means that the uncertainty of the estimates is correctly reported. That being said, we have also repeated the analysis by age group and compare the estimates with those of the combined model in the section titled statistical methods. This comparison shows that the two approaches generate similar estimates of counterfactual no-pandemic mortality, for the aforementioned reasons. Page 7, para 2, line 2: is it mid-February 2021? Please correct. Assuming the comment refers to the sentence starting with “We used data on weekly deaths from the start of time series through mid-February 2020 …”, “mid-February 2020” is correct as estimation of model parameters was based on data before the pandemic. Results: Do we have data on excess mortality for US states? May be interesting to add (unless it is part of a forthcoming study). Figures 2, 4 and 8, and the associated text, present results on excess mortality for US states. No data are presented regarding “lessons learned”. May be useful to study the social   Page 31 of 32 Wellcome Open Research 2022, 6:279 Last updated: 10 JUN 2022 determinants such as GDPs, income, and health systems. The countries studied here are all high-income although they differ in health care spending and health service infrastructure4. It would be a separate body of work, and a distinct paper, to quantify how various features of health system – from local public health and primary care to specialist care – may have affected the outcome of the pandemic. We now discuss this issue at the beginning of the Discussion section, with reference to a number of studies that have qualitatively and quantitative analysed the role of both health system resilience and early social distancing measures on the impacts of pandemic. Discussion: Please discuss the data presented and their limitations, and then focus on the implications of the study. We have reorganised the text as suggested. References 1. Venter ZS, Aunan K, Chowdhury S, Lelieveld J. COVID-19 lockdowns cause global air pollution declines. Proc Natl Acad Sci U S A 2020; 117(32): 18984-90. 2. Liu F, Page A, Strode SA, et al. Abrupt decline in tropospheric nitrogen dioxide over China after the outbreak of COVID-19. Sci Adv 2020; 6(28): eabc2992. 3. Forster PM, Forster HI, Evans MJ, et al. Current and future global climate impacts resulting from COVID-19. Nature Climate Change 2020; 10(10): 913-9. 4. Haldane V, De Foo C, Abdalla SM, et al. Health systems resilience in managing the COVID- 19 pandemic: lessons from 28 countries. Nat Med 2021; 27(6): 964-80. Competing Interests: No competing interests were disclosed.   Page 32 of 32