School closures have been one of the most popular government responses to the outbreak of COVID-19 caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Within just a month of the World Health Organization declaring COVID-19 a global pandemic in March 2020, 173 countries had closed schools, affecting nearly 85% of the world’s students. While the intent of these policies has been to protect the health of children, teachers, staff, and their communities, decisions to shutter schools have also come at a significant cost to society. Millions of children have lost out on the learning, physical health, and mental health benefits to attending classes, while parents have been forced to balance their regular work responsibilities with finding alternative childcare and educational options for their children.
The substantial negative consequences of closing schools led us to ask the following question: How effective are school closures at limiting the spread of COVID-19 in the first place? After all, if there is not much of a health benefit to shutting down schools—for instance, because children are less likely than adults to become infected or spread SARS-CoV-2—then governments may be imposing an unnecessary burden on families with children.
In our Nature Medicine article, we investigate this question by analyzing whether school closures in Japan were effective at reducing the number of COVID-19 cases in spring 2020. We do so by using detailed data on Japanese municipalities, which varied in whether they closed or opened their schools in the initial months of the pandemic.
While other studies have explored the correlation between school closures and COVID-19 cases before, our article pays special attention to evaluating whether this relationship is causal.
First, we control for dozens of factors that could explain an observed relationship between school closures and SARS-CoV-2 infection rates. Many studies simply compare whether communities that closed their schools have fewer COVID-19 cases than communities that did not over a given period, without accounting for other ways in which these communities differ. Ignoring these differences becomes especially problematic if these excluded characteristics influence both a community’s decision to close schools and the spread of COVID-19. For example, having an older population, higher population density, and more commuters may cause a community to both close its schools and have more COVID-19 cases, but this does not mean that school closures caused the spread of COVID-19. By accounting for the influence of a wide range of social, political, and environmental factors across Japanese municipalities in our analysis, we can therefore be more confident that we are measuring the specific impact of school closures on SARS-CoV-2 infection rates.
Second, we address a related issue known as “reverse causality.” Here is an example of the problem. Suppose a study finds that communities with open schools have fewer COVID-19 cases than communities with closed schools and then concludes that it is safe to keep schools open. However, it could be the case that communities with open schools continued to operate their schools exactly because they have fewer COVID-19 cases—in other words, the causal relationship may flow in the opposite direction. We address this issue by ensuring that we only compare Japanese municipalities that had similar levels of infection before making the decision to either open or close their schools.
Third, we account for other government responses to the pandemic. Many communities that closed their schools did so at the same time as they implemented other measures to reduce the spread of COVID-19. The difficulty for researchers is how to disentangle the effect of school closures from those of other government policies such as stay-at-home orders and prohibitions on gatherings. To overcome this challenge, we take advantage of the fact that in Japan, prefectures have the primary responsibility for formulating public health policies to combat the pandemic, but it is up to municipalities to decide whether to close their elementary and junior high schools. Thus, by controlling for prefecture-level policies, we can focus on the variation across municipalities and isolate the effect of school closures.
Finally, to estimate the causal effect of shutting down schools, we use a statistical method known as “matching.” In essence, we select pairs of municipalities that are very similar on the dimensions discussed above—e.g., municipality-level characteristics, prior trend in COVID-19 cases, and prefecture-level policies—and differ only in whether they closed their schools. We then calculate whether these two paired groups of municipalities differ in the number of COVID-19 cases.
We do not find any evidence that school closures in Japan helped to reduce the spread of COVID-19 in spring 2020. Among our pairs of similar municipalities, municipalities that closed their schools had very similar levels of COVID-19 as those that kept their schools open. In other words, municipalities that shut down their schools imposed additional costs on their residents without receiving the supposed benefit of limiting the spread of the pandemic.
To further confirm our main finding, we show that our results hold across a host of alternative approaches to modeling the relationship between school closures and COVID-19 cases. These robustness checks all point to the same conclusion: There is no evidence that shutting down schools had a significant impact on limiting the spread of COVID-19.
The reason why we do not find an effect may be complex. It could be because children are less susceptible to the effects of COVID-19 and less likely to transmit the virus to others. Or it could be because Japanese students participated in stringent mitigation strategies such as physical distancing, mask wearing, and enhanced hygiene. Or it could be because we analyze data from the initial months and first strain of COVID-19, but community transmission has gotten significantly worse since spring 2020 with the introduction of new variants (e.g., the alpha and delta variants).
While our article is not focused on identifying specific mechanisms, our results suggest that policymakers should be cautious in the future when considering whether to close schools to combat COVID-19, especially given the significant costs such policies can have for the well-being of children and parents. Our recommendation is that governments should monitor COVID-19 cases and school closures in each school district on a daily basis so that we can better evaluate the effectiveness of these policies.