The Future of Virtual Service Delivery: Lessons Learned from Research and Programmatic Experience

During the height of the pandemic in 2020, virtual service delivery was a necessity for many programs and initiatives PolicyLab researchers work on to protect the health and safety of children and families. Some have experience shifting their own programs to be entirely virtual, while others took the opportunity to research the impacts of shifting long-standing public health or clinical supports online to meet needs during the pandemic.

With COVID-19 resurgence impacting many communities across the U.S. again, we checked in with some of these researchers to hear more on what they’ve learned about virtual service delivery through their own experiences, and how they think those lessons can be applied to future efforts to meet families’ needs. You’ll see from their takeaways below, there are pros and cons to virtual engagement and lots still to uncover about the efficacy of delivering services virtually vs. in-person, but overall, with family input and thoughtful, evidence-based structure, there are many types of programs that may have a successful virtual future.

Dr. Jim Guevara

1. What are the top two takeaways from your research and/or programmatic experience related to virtual service delivery?

Thanks to a rapid research grant from the Leonard Davis Institute of Health Economics at the University of Pennsylvania, I was able to study the impact of safety net programs’ rapid shift to digital access and communications early in the pandemic. What I found was that there was a decrease in Early Intervention (EI) referral initiation post-pandemic in Philadelphia, but the percentage of families who were eligible and received services were similar pre- and post-pandemic. This was likely the result of perceptions that EI successfully communicated program changes, were flexible in scheduling visits and assisted families with transitioning to virtual visits. Parents were appreciative of these efforts but had concerns that virtual visits were associated with decreased child engagement and overall effectiveness compared to in-person visits pre-pandemic.

2. Based on your findings/experience, what considerations should program managers and service providers keep in mind when it comes to the future of virtual service delivery for children and their families?

As with any major program change, communication of impending changes to participants is crucial. Communication of changes and attention to the safety of participants and staff should be directed to not only participants but to those who refer to these programs, such as pediatricians. It’s also important that we allow families to choose the type of service delivery, as some families preferred in-person visits, while others preferred virtual visits or hybrid combinations.

Amy So, Dr. Molly Davis & Dr. Rebecca Kanine

1. What are the top two takeaways from your research and/or programmatic experience related to virtual service delivery?

In response to the pandemic, our team pivoted from testing the effectiveness of our depression prevention program, Interpersonal Psychotherapy-Adolescent Skills Training (IPT-AST), when delivered by school counselors to testing the efficacy of IPT-AST when delivered via telehealth to adolescents attending local schools. Feedback we have gathered thus far suggests that some adolescents may prefer telehealth and actually feel more comfortable engaging in treatment this way, while others found it challenging to avoid distracting text messages, social media alerts, and other interactions occurring in the home while participating in the groups. We also found that presenting group materials and collecting data entirely through electronic means, such as administering surveys via REDCap, maximized session engagement. Creating the role of “technology coordinator” on our team was vital for ensuring our telehealth IPT-AST groups ran smoothly.

Overall, telehealth has the potential to enhance access to high-quality behavioral health services for youth who may not otherwise receive such services. However, in implementing telehealth, we must be careful we do not inadvertently exacerbate the very disparities we are hoping to address. Telehealth delivery requires reliable internet connection and access to electronic devices. At a policy level, we must advocate for access to such resources to ensure our telehealth interventions reach children and families who are most in need.

Latifa Uqdah

1. What are the top two takeaways from your research and/or programmatic experience related to virtual service delivery?

The population of research participants from the programs I'm currently working on—such as the Young Child and Parent Project—have young children, multiple appointments, work schedules and other commitments they are trying to balance. The transition of connecting to services from a smart phone, tablet or laptop has been convenient for some participants, but I have also experienced some barriers with virtual services, such as participants lacking WIFI or an appropriate device for connecting to services or forgetting to connect on time.

I think it could be helpful for families to continue to receive some form of tutorial for using the specific video conferencing program through which their virtual services will be provided. Families use various programs for conferencing and are less familiar with some compared to others. Auto email/text reminders have also been helpful to families who have scheduled services.

Dr. Ariel Williamson

1. What are the top two takeaways from your research and/or programmatic experience related to virtual service delivery?

Sleep Well! is a brief behavioral sleep intervention adapted to treat insomnia (difficulty falling/staying asleep) and insufficient sleep among toddlers and preschoolers presenting to primary care. At the start of the pandemic, we transitioned from in-person, primary care-based visits to video and telephone visits only. Although our current randomized controlled trial of Sleep Well! is still underway, families who have completed the program have reported high levels of acceptability for virtual services. So far, 91% of families agreed or strongly agreed that they felt comfortable completing the program by video and/or telephone (9% felt neutral), while 84% agreed or strongly agreed that these virtual services were just as helpful as in-person care (16% felt neutral).

Based on participant feedback and our research team members’ experiences, flexibility in virtual service delivery and resources to promote access to such services are important. Families have reported that being able to schedule visits without having to arrange transportation or account for travel to a primary care site has benefitted their ability to engage in Sleep Well!. This feedback aligns with our qualitative research on adaptations needed for sleep interventions tailored for primary care and for families of lower-income backgrounds. Being able to select from a combination of video and telephone visits and having support from the research team to problem-solve any in-the-moment issues with the virtual platform has also helped to increase access to this program. A critical next step is to identify whether Sleep Well! is efficacious for treating child sleep problems when delivered in this virtual format, as this information can inform dissemination efforts and potentially extend the reach of behavioral sleep interventions.

Dr. Joanne Wood

1. What are the top two takeaways from your research and/or programmatic experience related to virtual service delivery?

After the pandemic forced us to suspend our in-person PriCARE group positive parenting classes for parents of children 2 to 6 years old, we switched to offering smaller virtual classes. More than ever, parents of young children were seeking help with child behavioral problems at home. Initially, some parents declined the virtual option and expressed an interest in waiting for in-person sessions to resume. As the pandemic continued, however, more and more parents became open to virtual classes and the demand quickly outpaced our capacity. Parents reported high rates of satisfaction with the virtual program, and we saw increases in attendance rates with virtual delivery. Our team, however, worried that maintaining parental engagement and fostering peer support was more challenging virtually. Additional work will be needed to fully assess the efficacy of the virtual format compared to in-person classes.

Program managers and providers need to be prepared to make significant changes to the delivery of services and be creative in order to meet the needs of families. For example, with the switch to virtual service delivery we were no longer able to provide onsite child care, which was key to supporting parents in attending the in-person groups. In order to facilitate parents’ participation without being distracted by their children, we moved our classes to late evening/nighttime when many younger children are in bed. We also provided parents with activity sets, containing a new activity for each class to engage children who may be still awake. Also, while virtual delivery may offer some benefits such as eliminating transportation barriers and making it easier for some families to participate, the potential downsides of virtual delivery such as decreased engagement will also need to be carefully considered when making decisions regarding the continuation of virtual services.

Katie Wu

1. What are the top two takeaways from your research and/or programmatic experience related to virtual service delivery?

My team and I researched the strategies and experiences of delivering virtual community health worker services to connect patients and families with medical and social services. In general, community health workers and program participants found it straightforward to adapt to regular phone calls instead of home or community visits, highlighting that connecting virtually felt safer than meeting in-person due to the pandemic. Still, some program participants were hesitant when their community health worker first called them because they were not sure whether to trust that the program was real and not a scam. Overall, community health workers and program participants reported that they were able to build trust and rapport virtually by phone, text, email and video.

The first key theme that emerged from our research was the need for flexibility when delivering services virtually, especially in the scheduling and format of check-ins with participants (e.g., call, text, video, email) and walking participants through how to use new technologies (e.g., signing online consent forms, using video software). Another key theme was the importance of intentionally building trust with participants, especially with limited or no in-person meetings. Opportunities to build trust with participants virtually include giving concrete explanations to patients and families about how and when to expect someone to contact them after they are referred to a program and highlighting any links a program has to a patient’s or family’s existing services (e.g., community health workers highlight that they work closely with a participant’s primary care practice).

Amy So is a former clinical research coordinator at PolicyLab.

Rebecca Kanine, PhD, is a former research scientist at PolicyLab.

Latifa Uqdah, MS, is a former research assistant at PolicyLab.