Text4baby Data and Evaluation
Text4baby’s power lies in its ability to get the most essential health information to mothers in need quickly and easily using a technology they regularly use and rely on. Over 91% of Americans own a cell phone and 81% of cell users send or receive text messages.1 Text4baby is a data-driven initiative. Staff routinely monitor and analyze a wide array of data collected by the program, including data on enrollment, completion and cancellation, program dosage, user-reported reason for cancellation, referral source, and descriptive data from various questions asked through the mobile platform. This data is reviewed to inform and improve program promotional, outreach, and product strategies. Additionally, Text4baby provides partners with access to real-time enrollment data to support them in understanding the impact of outreach initiatives on enrollment. Below are some key highlights from Text4baby’s ongoing data and evaluation efforts.
Text4baby is Reaching its Target Population
- Text4baby is reaching individuals early in their pregnancy: Of the participants who signed up to receive pregnancy messages, over 47% enrolled during the first trimester.
- Text4baby is reaching women in high-poverty areas: At enrollment, a higher percentage of Text4baby participants live in zip codes with the highest levels of poverty compared to the overall U.S. distribution. Click here to view a graph representing these findings.
- CSUSM/UCSD Text4baby survey respondents from low income households: Nearly half of the participants who participated in the California State University San Marcos National Latino Research Center and University of California, San Diego (CSUSM/UCSD) evaluation of Text4baby reported their household income was $20,000 or less. 2
Text4baby is Well Received by Participants
- 99% of WIC participants in an Emory University study (baseline n=468) had no concerns about enrolling in Text4baby; 95% reported the enrollment process was easy; 92% regularly read Text4baby messages; and 88% planned to continue to use Text4baby.3
- 93% of participants who responded to a Text4baby survey said they would refer Text4baby to a friend (n=42,981) and rated the helpfulness of the service a 7.7 out of 10 (n=66,615).
- A short survey conducted with patients at Carilion Clinic in Roanoke, Virginia who enrolled in Text4baby for six months or more found that 92% of women who continued to use the service (49 of 53) were satisfied with the service.4
- The average satisfaction rating for participants who participated in the CSUSM/UCSD evaluation was 8.5 out of 10, with Spanish-speaking participants reporting a higher level of satisfaction compared to English-speaking participants.5
Evidence of Knowledge and Behavior Change
- Increasing Health Knowledge and Preparedness: 74% of total respondents who participated in the CSUSM/UCSD evaluation reported Text4baby messages informed them of medical warning signs they did not know.2 A George Washington University-led randomized evaluation found that Text4baby mothers were nearly three times more likely to believe that they were prepared to be new mothers compared to those in the no exposure control group.6
- Appointment Attendance: 65% of total respondents who participated in the CSUSM/UCSD evaluation reported Text4baby helped them remember an appointment or immunization that they or their child needed.2 Additionally, the Alliance of Chicago Community Health Services (network of Federally Qualified Health Centers) found the percentage of missed appointments was lower among Text4baby enrollees (n=917) compared to non-enrollees (n=1,647) when comparing the two patient populations at one point in time (11% vs. 17% missed appointments, respectively).7
- Facilitating Interaction with Health Providers and Improving Access to Health Services: 67% of total respondents who participated in the CSUSM/UCSD evaluation reported they talked to their doctor about a topic they read on a Text4baby message and 40% reported they called a number for a service they received from Text4baby, with a higher percentage of uninsured respondents reporting they called a service number.2
External Evaluations of Text4baby
Three external evaluations of Text4baby are currently underway, including:
- Funded by the U.S. Department of Health and Human Services, Mathematica Policy Research is conducting a mixed-method process and outcome evaluation of Text4baby. Evaluation methods include national stakeholder interviews, the implementation of a pre-post consumer survey, electronic health record abstraction, key informant interviews, and consumer focus groups in four communities.
- Madigan Army Medical Center in Takoma, Washington in collaboration with George Washington University is conducting a randomized controlled trial with a target sample of 1,000 pregnant women seeking care at Madigan. Study methods include four surveys designed to measure changes in a number of knowledge and behavior outcomes for Text4baby participants compared to the control group.
- The University of Maryland School of Public Health, Herschel S. Horowitz Center for Health Literacy is conducting two separate evaluation efforts, including (1) a small pilot to assess receptivity to Text4baby messages and knowledge, awareness and behavioral outcomes among Text4baby participants and non-participants; and (2) an analysis of Text4baby messages based on health communication and health behavior theory in an effort to identify how Text4baby messages align with theories.
Other ongoing research involving Text4baby includes:
- Researchers from St. Louis University are conducting a study to assess the impact of a text reminder intervention on adherence to a diabetes care regimen and glycemic control. Preliminary results presented at the Society of Maternal and Fetal Medicine’s annual conference show a significant difference in average glucose values within goal between the Text4baby control group and the no-text control group (73% vs 20%, p<0.01). Recruitment for the study is ongoing.
Internal Research and Evaluation Efforts
Text4baby regularly consults with experts to guide our internal research and evaluation efforts. Current and future research and evaluation efforts include:
- Text4baby worked with the California Department of Health and Immunize Nevada to implement a childhood vaccination module pilot project in March 2013. The goal of the project was to gain insight on self-reported childhood vaccination status and well baby visit attendance among Text4baby participants. Preliminary results show higher self-reported appointment attendance among participants who received educational and appointment reminder messages compared to those who did not receive them, and high rates of self-reported hepatitis B vaccination in the pilot project sample. Click here to learn more about the results of the immunization module.
- In collaboration with CSUSM/UCSD, Text4baby is conducting a phone survey, promoted via SMS, covering topics assessed in the CSUSM/UCSD evaluation with a large random sample of participants across the country.
- In October 2012, Text4baby implemented a flu module designed to gain insight around whether providing appointment reminders and tailored educational messaging via text improves self-reported vaccination coverage among Text4baby participants. The results were recently presented at the 2013 mHealth Summit. Click here to read the abstract of the presentation.9
- Two additional routine survey questions were implemented by Text4baby in March 2013 to gain a better understanding of participant help-seeking behavior and patient engagement.
- In partnership with the Centers for Medicare and Medicaid Services, Text4baby implemented a Medicaid module. The goal of the module was to identify the health insurance status of Text4baby participants and provide information on Medicaid/CHIP to participants who indicated they were uninsured. Preliminary results suggest messaging sent to uninsured women may encourage Medicaid/CHIP application among pregnant and new mother Text4baby participants. Click here to learn more about the results of the Medicaid module. Additionally, the Kaiser Family Foundation (KFF) recently conducted interviews with Text4baby participants to better understand the impact of the Medicaid Module. Findings are highlighted in their February 2014 Issue Brief. 10
Click here for a factsheet summarizing Text4baby's impact.
Access to Data for Text4baby Outreach Partners
The National Healthy Mothers, Healthy Babies Coalition (HMHB) and Voxiva are pleased to support access to data for organizations who have signed a Memorandum of Understanding as Outreach Partners for Text4baby. The process for partners to access data is as follows:
1. Complete MOU found here.
2. During your intake call with HMHB staff, you will be informed about next steps for data access, which will include completing a Data Use Agreement.
3. Once the DUA is completed, HMHB staff will work with Voxiva to create log-in credentials to give you access to real-time, zip-based enrollment data in your state.
4. HMHB hosts regular webinars to train you on how to access and use the enrollment database. A staff member will contact you regarding training.
Conducting Research and/or Publishing on Text4baby
If you are interested in conducting research on Text4baby, please email email@example.com with details about your request.
1. Duggan, M. (2013). Cell Phone Activities 2013. Pew Research Center. Available from: http://www.pewinternet.org/2013/09/19/cell-phone-activities-2013/
2. California State University San Marcos National Latino Research Center and University of California San Diego (2012). Maternal and Newborn Health: Text4baby San Diego. Evaluation Overview: October 2011-October 2012. http://www.csusm.edu/nlrc/documents/report_archives/Text4Baby_SanDiego_Evaluation_Overview.pdf. Data collected via three surveys implemented October 2011- October 2012. Total sample size=626. Total respondents who provided income level=480.
3. Gazmararian, J., Elon, L., Yang, B., Graham, M., Parker, R. (2013). Text4baby Program: An Opportunity to Reach Underserved Pregnant and Postpartum Women? Maternal Child Health Journal. Abstract available: http://www.ncbi.nlm.nih.gov/pubmed/23494485.
4. Kaleka, A., Olsen, R., & Sweet, M. (2012, April 26). Utilization of Text4baby to Improve Maternal and Infant Outcomes with an Interdisciplinary Team. Presented at STFM Annual Conference, Seattle, Washington. Available from: http://www.fmdrl.org/index.cfm?event=c.getAttachment&riid=6110.
5. California State University, San Marcos (2011). San Diego Researchers First to Report Positive Impact of Text4Baby Program [press release]. Retrieved from: https://www.text4baby.org/templates/beez_20/images/HMHB/SD_press_release.pdf. Totalsample size for first survey = 122.
6. Evans, W., Wallace, J., and Snider, J. Pilot Evaluation of the text4baby Mobile Health Program, BMC Public Health, 12. Available: http://www.biomedcentral.com/1471-2458/12/1031/abstract.
7. Although the Alliance findings are not the result of rigorous research (and thus not statistically significant), the trend is positive and will continue to be monitored.
8. Centers for Disease Control and Prevention. Pregnant Women and Flu Shots, Internet Panel Survey, United States, November 2012. Available: http://www.cdc.gov/flu/pdf/fluvaxview/pregnant-women-2012.pdf.
9. Abstract from Research Symposium at Dec 2013 mHealth Summit. “Findings from an Interactive Module Encouraging Influenza Vaccination among Text4baby Participants.” Accessible at: http://www.mhealthsummit.org/program-details/findings-interactive-module
10. Kaiser Family Foundation. Feb 2014. “Profiles of Medicaid Outreach and Enrollment Strategies: Using Text Messaging to Reach and Enroll Uninsured Individuals into Medicaid and CHIP” Accessible at: http://kff.org/medicaid/issue-brief/profiles-of-medicaid-outreach-and-enrollment-strategies-using-text-messaging-to-reach-and-enroll-uninsured-individuals-into-medicaid-and-chip/