Conclusion¶
The goal of the project was to determine how the different COVID-19 policies affect health and economic factors. The COVID policies implemented by Democratic led states and Republican led states were considerably different, and often fell along party-lines. The former can be categorized by more restrictive policies like mask-wearing and social distancing while the latter can be categorized by less restrictive policies like hasty re-opening initiatives and less abundant mask-wearing and social distancing measures, as evidenced by Governor issued directives and orders. Specifically, we analyzed the testing, positives, and death rates of Republican and Democratic states, the 4 regions of the United States, and California vs. Florida.
We first compared the averages of total testing, positives, percent positive, and deaths in Democratic and Republican states as of 3/07/2021 using T-Tests. In turn, we found that Democratic states had significantly higher testing rates, Republican states had significantly higher positive rates, and a higher percentage of tests returned positive. However, the difference in average death rates was not significant. We found similar results when we compared testing, positive, and death rates of Republican and Democratic states on a day by day basis.
We also compared California, Florida, and the national average in specific time periods before and after Florida began phase 1 reopening. This included the entire time frame, the time frame only after Florida began stage 1 reopening, and the 6 months directly after Florida began stage 1 reopening. California had a significantly higher average testing rate than Florida and the national average in the entire time frame, and after phase 1 reopening, but not the 6 months after Florida began phase 1 reopening. Florida had a significantly higher positive rate than Florida and the national average in only the 6 months after phase 1 reopening. Florida’s average death rate was higher than California and the national average in only the six month period after phase 1 reopening.
To better understand what role governmental policy played throughout the COVID-19 pandemic, we ran a number of linear (OLS) regressions - including one as a difference in differences (DD) model - measuring changes in positive test rates between CA and FL while controlling for a handful of other explanatory variables (e.g. tests administered).
Across the board, the coefficient for testing rates was a statistically significant 0.00 - meaning the model predicts there is no correlation between increased testing and swings in the positive test rate. Not statistically significant, however, was the coefficient of being in either Florida or California when considering the data in its entirety. This ultimately defied expectations, as the model suggests that - despite vast policy differences - being under the rule of one state or the other doesn’t necessarily lead to changes in the positive test rate. We went on to reapply this model to a 2021-only dataset (to better assess more recent developments) and found a similarly unexpected result: a statistically significant prediction that Florida’s 2021 approach led to a greater decrease in positive tests than that of California. These findings ultimately set a foundation for the contrary notion that the (economically costly) COVID-19 restrictions imposed by states may not have yielded positive results as intended.
To account for the effect of shared changes over time, and to address the heterogeneity between California and Florida beyond the use of control variables, we chose to incorporate a Diff-in-Diff model into our last linear regression. Essentially, we trained this model to measure the underlying differences between CA and FL in the early stages of the pandemic; this ultimately afforded us a baseline trend that could predict what would have happened in a ‘treated’ state based off of its known relationship to an ‘untreated’ state. We then recognized Florida’s 9/25 complete reopening as a major change and valid ‘treatment’ in our natural experiment. The results from this model were particularly interesting in lieu of our original question, as isolating a single major policy shift led us to find that Florida actually fared better at a statistically significant level in this instance.
We also compared the averages of total testing, positives, percent positive, and deaths for the four regions of the United States using ANOVA and Tukey tests. We found that the Midwestern and Northeastern states had statistically significant differences in testing rates and percent of tests that returned positive. This makes sense since the Midwest is very rural and COVID tests are less accessible. The Northeastern testing rate and death rates were significantly higher than those of the West. The difference in death rates may be because the COVID pandemic originated in New York and hit the northeastern states the worse before eventually reaching other parts of the country. We are unsure why the West’s testing rate is lower than the Northeastern one.
When we compared the difference in unemployment rates of Democratic and Republican states, we found that the difference increased after the COVID pandemic, meaning that Republican states had significantly lower average unemployment rates than Democratic states after the national emergency of the COVID-19 pandemic declared on March 13, 2020. Similarly, Florida also had a significantly lower unemployment rate average than California in this time period.
Overall, there is an association between less restrictive Republican states and lower testing rates, higher positives, and lower unemployment rates. The difference in death rates is generally insignificant. However there are many lurking variables that we can’t account for that have influenced the data.
Data Limitations¶
We are foremostly limited by the data sets that were available to us. It is likely that the true COVID-19 case and death counts are higher than what was recorded, and there may be reporting discrepancies between states as well.
We sought to incorporate a handful of other metrics in our research, but a lack of publicly accessible data (and data in general, given the recency of our topic) limited the overall breadth of resources at our disposal.
Incorporating economic measures by state also proved troublesome - as there are few ways to reliably assess state-level economies in real time. Unemployment proved to be a decent metric, but a day-to-day measure of household income and/or poverty rates would’ve been ideal (annual reports on such metrics were available, but the data was too infrequent to draw conclusions from).
It is difficult to control for all lifestyle differences between FL and CA or between Republican or Democrat affiliated states which could otherwise affect the spread of COVID-19 (i.e. feasibility of being outdoors where spread is less likely, access to private transportation, etc.) Republican states tend to be more rural and may have less accessibility to testing. In that manner, an innate bias exists in the data. There is no way to measure the population’s willingness to follow COVID guidelines, as many people chose to ignore them. We also cannot account for interstate or international travel. In the United States, the COVID pandemic started in the mostly Democratic Northeast and spread to the rest of the country. This is probably why the northeast region has a higher total death rate than other regions of the country and it may have skewed the average death rates of all Democratic states, This represents another inherent bias that we did not account for.
We determined the states’ political affiliation based on the results of the 2020 presidential election. A states’ covid policies are more influenced by the political affiliation of its governor, but the presidential election results generally show if the state’s population leans more Democratic or Republican. However if a state recently changed political majorities, its data may contribute to the opposite political affiliation that it belongs to. Also it is possible for a state’s majority political affiliation to have changed in the time frames we measured. Overall this method of determining political affiliation may not represent swing states or mixed populations well.
The Covid Tracking Project, the source of one of our state and national COVID data sets, ended its collection on March 7, 2021, so we only have COVID data up to that point. The website for the unemployment data from the United States Department of Labor only allowed us to download the information for one state at a time. Because of this, we only analysed the weekly unemployment rates of California and Florida. From the U.S. Bureau of Labor statistics, we were able to get the monthly unemployment rates of all 50 states from March 2011-2021, which is what we used to estimate the average unemployment rates of Democrat and Republican states.
We also noticed an aberration in the data where higher testing rates were associated with higher death rates. When we checked the documentation of the Covid Tracking Project, we found they only reported the deaths of people who had tested positive for COVID or who showed symptoms and were likely infected with COVID. Obviously, a state’s testing rate would influence the number of confirmed deaths reported. Also, states may have different criteria for determining probable COVID deaths which could lead to underrepresentation.
We also wanted to graph the percentage of tests that were positive on the day to day basis but were unable to do so because COVID tests take 1-7 days to process, and positveIncrease / TotalTestsIncrease would not accurately reflect the true percentage of tests that were positive on a given day. In early 2020, it was often that new positives were reported on days when no new tests were reported which would cause a division by zero error that would affect later calculations.
