PolicyLab

I didn’t know I would become a biostatistician when I was studying biology at Wuhan University in Wuhan, China, 15 years ago—just like I would never know that the city of Wuhan would become world famous as the origin site of a pandemic that changed history. Having studied and lived in Wuhan for five years, many of my friends are from or live there. Due to this experience, plus my job as a public health researcher, this pandemic has impacted me differently.

My story began in December 2019, when a few of my friends in Wuhan started to talk about strange cases of a pneumonia-like illness on social media. At that point, we didn’t know the disease, later known as COVID-19, was going to kill more than 3.8 million people worldwide. In January 2020, for the first time, I learned that a friend’s mother had symptoms of COVID-19. She waited for more than six hours in the outpatient department of a hospital in Wuhan for a confirmed diagnosis and was later sent home to quarantine with a prescription for medicine that had been out of stock for weeks in the city.

Her disease turned worse quickly, and without any hope of finding a hospital bed her family needed to seek other options. Finally, they were able to purchase plane tickets to another city and they planned to go there for health care. However, just before their flight was supposed to depart from Wuhan, the city enacted a mandatory lockdown order. They were stuck once again.

The photos above, supplied by Jing Huang, show Wuhan University during the COVID-19 lockdown in 2020.

Over the past 16 months, we have heard of, read or even experienced many stories like this. It’s easy to feel powerless and stuck in the face of COVID-19, but we can make a difference. As a biostatistician, working with a team on a county-level COVID-19 forecasting model, called COVID-Lab, has been the most rewarding and life-changing experience of my career. It is also one of the few things that I have done in my life that was not driven by deadlines, but by an inner-human drive to defend and protect.

Between April 2020 and May 2021, we disseminated weekly projections through COVID-Lab that forecasted case transmission for as many as 821 counties with active outbreaks. From data exploration and model design to programing and analyzing our findings, I, as well as the team, also occasionally felt powerless and stuck at many crossroads like my friend’s family as we stumbled along the journey of modeling a pandemic. But we also learned valuable lessons. Here, I want to share my top three lessons learned about using biostatistics to model a pandemic with you:

• Lesson 1: Trust your data like you trust your partner. As a biostatistician, my training has been all about being skeptical and working with uncertainties. It’s easy for us to lose confidence when seeing many errors in the COVID-19 data, including measurement errors and coding errors due to varying criteria for testing, delayed reporting of cases and, sometimes, human mistakes. However, data are footprints of evidence. As more data are collected, evidence also accumulates—it’s just buried under errors that need to be extracted using appropriate statistical tricks. Integrating different data elements also helps reveal the evidence from the noise. One of the key advantages of our model is that we were able to utilize information from data on case counts, testing, hospitalizations, geographic variables and policies to inform our modeling projections.
• Lesson 2: it’s never too late to update a model. The dynamics of COVID-19 transmission have evolved quickly over the past year. We found that a model that worked well in early 2020 gradually lost accuracy as people changed behaviors over time to adjust to a new normal. A social distancing variable that measured frequency of visits to nonessential business was the strongest predictor of transmission rates in April 2020, but not in January 2021. Therefore, we updated our model consistently in order to look for new predictors and to understand changes.
• Lesson 3: validations can be more useful than theories in practice. More than one year into the pandemic, we are still learning new things all the time about this virus. There are limited theories to tell us what are the most appropriate statistical models to use when creating case count projections. Instead, we learned from what occurred and compared our projections with the reality to inform updates to or validate our model. You can read more about how we validated our model here.

This pandemic has put biostatistics under the spotlight, but statistics means more than numbers. Rigorous mathematical models, carefully assessed assumptions and interpretable results for the public have always been the goals of the COVID-Lab modeling team. Our model is also applicable to other infectious diseases, so it can be used to manage future pandemics. We are grateful to contribute a small bit during this difficult time, and deeply believe the lessons we learned and efforts made will pave the way to an improved response in the future.