Text4baby Data and Evaluation
Text4baby’s power lies in its ability to get the most essential health information to mothers in need quickly and easily using a technology they regularly use and rely on. Over 91% of Americans own a cell phone and 81% of cell users send or receive text messages. Text4baby is a data-driven initiative. Staff routinely monitor and analyze a wide array of data collected by the program, including data on enrollment, completion and cancellation, program dosage, user-reported reason for cancellation, referral source, and descriptive data from various questions asked through the mobile platform. This data is reviewed to inform and improve program promotional, outreach, and product strategies. Additionally, text4baby provides partners with access to real-time enrollment data to support them in understanding the impact of outreach initiatives on enrollment. Below are some key highlights from text4baby’s ongoing data and evaluation efforts.
Text4baby is Reaching its Target Population
- Text4baby is reaching individuals early in their pregnancy: Of the participants who signed up to receive pregnancy messages, over 47% enrolled during the first trimester.
- Text4baby is reaching women in high-poverty areas: A higher percentage of text4baby participants live (or lived upon enrollment) in zip codes with the highest levels of poverty compared to the overall U.S. distribution.
- CSUSM/UCSD text4baby survey respondents from low income households: Nearly half of the participants who participated in the California State University San Marcos National Latino Research Center and University of California, San Diego (CSUSM/UCSD) evaluation of text4baby reported their household income was $20,000 or less. 1
Text4baby is Well Received by Participants
- 99% of WIC participants in an Emory University study (baseline n=468) had no concerns about enrolling in text4baby; 95% reported the enrollment process was easy; 92% regularly read text4baby messages; and 88% planned to continue to use text4baby.2
- 93% of participants who responded to a text4baby survey said they would refer text4baby to a friend (n=26,650) and rated the helpfulness of the service a 7.7 out of 10 (n=38,090).
- A short survey conducted with patients at Carilion Clinic in Roanoke, Virginia who enrolled in text4baby for six months or more found that 92% of women who continued to use the service (49 of 53) were satisfied with the service.3
- The average satisfaction rating for participants who participated in the CSUSM/UCSD evaluation was 8.5 out of 10, with Spanish-speaking participants reporting a higher level of satisfaction compared to English-speaking participants.4
Participants Embrace Text4baby as an Interactive Tool
- Engaged and Willing to Respond to Text Surveys: In October 2011, 31% (29,316 of 96,070) of participants responded to a text asking about their intention to get a flu shot.
- Expressed Interest In Receiving a Text Reminder: 55.7% of participants who indicated they were planning to get a flu shot and responded to a text asking if they wanted to receive a reminder (n=4,578) replied “yes.”
- High Response to Sensitive Topics: Nearly three quarters (73%) of respondents not planning to get the shot (n=8,841) were willing to provide a reason when asked, “why not?” via text. Concerns reported by participants are consistent with a Centers for Disease Control and Prevention survey of pregnant women and illustrate the potential of text4baby to obtain health information from hard-to-reach populations.7
Evidence of Knowledge and Behavior Change
- Increasing Health Knowledge and Preparedness: 74% of total respondents who participated in the CSUSM/UCSD evaluation reported text4baby messages informed them of medical warning signs they did not know. A George Washington University-led randomized evaluation found that text4baby mothers were nearly three times more likely to believe that they were prepared to be new mothers compared to those in the no exposure control group.5
- Appointment Attendance: 65% of total respondents who participated in the CSUSM/UCSD evaluation reported text4baby helped them remember an appointment or immunization that they or their child needed. Additionally, the Alliance of Chicago Community Health Services (network of Federally Qualified Health Centers) found the percentage of missed appointments was lower among text4baby enrollees (n=917) compared to non-enrollees (n=1,647) when comparing the two patient populations at one point in time (11% vs. 17% missed appointments, respectively).6
- Facilitating Interaction with Health Providers and Improving Access to Health Services: 67% of total respondents who participated in the CSUSM/UCSD evaluation reported they talked to their doctor about a topic they read on a text4baby message and 40% reported they called a number for a service they received from text4baby, with a higher percentage of uninsured respondents reporting they called a service number.
External Evaluations of Text4baby
Three external evaluations of text4baby are currently underway, including:
- Funded by the U.S. Department of Health and Human Services, Mathematica Policy Research is conducting a mixed-method process and outcome evaluation of text4baby. Evaluation methods include national stakeholder interviews, the implementation of a pre-post consumer survey, electronic health record abstraction, key informant interviews, and consumer focus groups in four communities.
- Madigan Army Medical Center in Takoma, Washington in collaboration with George Washington University is conducting a randomized control trial with a target sample of 1,000 pregnant women seeking care at Madigan. Study methods include four surveys designed to measure changes in a number of knowledge and behavior outcomes for text4baby participants compared to the control group.
- The University of Maryland School of Public Health, Herschel S. Horowitz Center for Health Literacy is conducting two separate evaluations efforts, including (1) a small pilot to assess receptivity to text4baby messages and knowledge, awareness and behavioral outcomes among text4baby participants and non-participants; and (2) an analysis of text4baby messages based on health communication and health behavior theory in an effort to identify how text4baby messages align with different health communication theories.
Internal Research and Evaluation Efforts
Text4baby regularly consults with experts to guide our internal research and evaluation efforts. Current and future research and evaluation efforts include:
- Text4baby worked with the California Department of Health and Immunize Nevada to implement a childhood vaccination module pilot project in March 2013. The goal of the project was to gain insight on self-reported childhood vaccination status and well baby visit attendance among text4baby participants. Preliminary results show higher self-reported appointment attendance among participants who received educational and appointment reminder messages compared to those who did not receive them, and high rates of self-reported hepatitis B vaccination in the pilot project sample. Click here to learn more about the results of the immunization module.
- In collaboration with CSUSM/UCSD, text4baby is conducting a phone survey, promoted via SMS, covering topics assessed in the CSUSM/UCSD evaluation with a large random sample of participants across the country.
- In October 2012, text4baby implemented a flu module designed to gain insight around whether providing appointment reminders and tailored educational messaging via text improves self-reported vaccination coverage among text4baby participants. Data analysis is currently underway.
- Two additional routine survey questions were implemented by text4baby in March 2013 to gain a better understanding of participant help-seeking behavior and patient engagement.
- In partnership with the Centers for Medicare and Medicaid Services, text4baby implemented a Medicaid module. The goal of the module was to identify the health insurance status of text4baby participants and provide information on Medicaid/CHIP to participants who indicated they were uninsured. Preliminary results suggest messaging sent to uninsured women may encourage Medicaid/CHIP application among pregnant and new mother text4baby participants. Click here to learn more about the results of the Medicaid module.
Access to Data for Text4baby Outreach Partners
The National Healthy Mothers, Healthy Babies Coalition (HMHB) and Voxiva are pleased to support access to data for organizations who have signed a Memorandum of Understanding as Outreach Partners for text4baby. The process for partners to access data is as follows:
1. Complete MOU found here.
2. During your intake call with HMHB staff, you will be informed about next steps for data access, which will include completing a Data Use Agreement.
3. Once the DUA is completed, HMHB staff will work with Voxiva to create log-in credentials to give you access to real-time, zip-based enrollment data in your state.
4. HMHB hosts regular webinars to train you on how to access and use the enrollment database. A staff member will contact you regarding training.
Conducting Research and/or Publishing on Text4baby
If you are interested in conducting research on text4baby, please email email@example.com with details about your request.
1. California State University San Marcos National Latino Research Center and University of California San Diego (2012). Maternal and Newborn Health: Text4baby San Diego. Evaluation Overview: October 2011-October 2012. http://www.csusm.edu/nlrc/documents/report_archives/Text4Baby_SanDiego_Evaluation_Overview.pdf. Data collected via three surveys implemented October 2011- October 2012. Total sample size=626. Total respondents who provided income level=480.
2. Gazmararian, J., Elon, L., Yang, B., Graham, M., Parker, R. (2013). Text4baby Program: An Opportunity to Reach Underserved Pregnant and Postpartum Women? Maternal Child Health Journal. Abstract available: http://www.ncbi.nlm.nih.gov/pubmed/23494485.
3. Kaleka, A., Olsen, R., & Sweet, M. (2012, April 28). Utilization of Text4baby to Improve Maternal and Infant Outcomes with an Interdisciplinary Team. Seattle, Washington. Available: http://www.fmdrl.org/index.cfm?event=c.accessResources&rid=3850.
4. California State University, San Marcos (2011). San Diego Researchers First to Report Positive Impact of Text4Baby Program [press release]. Retrieved from: https://www.text4baby.org/templates/beez_20/images/HMHB/SD_press_release.pdf. Totalsample size for first survey = 122.
5. Evans, W., Wallace, J., and Snider, J. Pilot Evaluation of the text4baby Mobile Health Program, BMC Public Health, 12. Available: http://www.biomedcentral.com/1471-2458/12/1031/abstract.
6. Although the Alliance findings are not the result of rigorous research (and thus not statistically significant), the trend is positive and will continue to be monitored.
7. Centers for Disease Control and Prevention. Pregnant Women and Flu Shots, Internet Panel Survey, United States, November 2012. Available: http://www.cdc.gov/flu/pdf/fluvaxview/pregnant-women-2012.pdf.