This study followed the standard guidelines for reporting quasi-experimental studies using the Transparent Reporting of Evaluations with Nonrandomized Design/Quasi-Experimental Study Design (TREND).
We confirm that all methods were carried out in accordance with relevant guidelines and regulations and in accordance with the declaration of Helsinki.
Study type
A quasi-experimental study with retrospective records review was conducted at some selected health facilities in the Savanah Region between January 2019 and May 2019. A data extraction tool was used to extract data from antenatal, delivery, and postnatal care registers in the Sawla-Tuna-Kalba (intervention site) and Bole (comparison) districts, all in the Savannah Region.
Study setting
The study was conducted at Kalba, Sawla, Tuna, and Gindabo Health Centres in the Sawla-Tuna-Kalba district as the study sites and Mankuma, Bole, Bamboi, and Mwandari Health Centres in the Bole District of the Savannah Region of Ghana as the comparison sites. The control district was selected purposively based on certain similar characteristics it shares with the intervention district. The two districts were one until 2004 when the Sawla-Tuna-Kalba district was carved out of the Bole district. Thus, they are twin districts. Both districts share boundaries with La’cote Dvore, they are both cosmopolitan districts with the inhabitants in both made up of Gonjas, Brifors, Vaglas, Dagaabas, Mos, and Lobis. The main economy within these two districts is farming. The settlements and culture of these two districts are similar.
Sawla-Tuna-Kalba district is one of the seven districts in the Savannah Region of Ghana. It has a total population of 125,525 (projections from the 2010 Population and Housing Census) with 278 communities. It shares boundaries to South, with Bole District, to the North, with Wa West District in the Upper West Region, West Gonja District to the east, and to the west La cote d’Ivoire and Burkina Faso. The district has 22 health facilities with one polyclinic, seven Health Centres, and 14 CHPS facilities. Two of these facilities are privately owned with three under the Christian Health Association of Ghana (CHAG) and the rest under the Ghana Health Service.
The Bole district on the other hand is in the western corridor of the Savannah region of Ghana. It has a population of 77,422 (projections from the 2010 Population and Housing Census) with 187 communities. It shares borders with La cote d’Ivoire to the West, Tain district to the south, West Gonja Municipal and Central Gonja District to the east, and Sawla-Tuna-Kalba District to the North. The major towns in the district are Bole, Bamboi, Mwandari, Mankuma, Jama, and Tinga with Bole as the district capital. The district has one District Hospital, seven Health Centres (one owned by CHAG), and eighteen CHPS facilities.
The control health facilities within the districts were purposively selected based on certain features they share with the intervention health facilities. Amongst them is the cadre of staff such as Physician assistants, Midwives, community health nurses, general nurses, etc. Also, the locations of the facilities were taken into consideration, e.g. Kalba health center in the intervention district share boundary with La cote d’Ivoire likewise Mwandari Health Centre in the comparison district. Consideration was also given to the level of prescription of the facility, that is, all the health facilities (intervention and control) were at the same level of standard classification of the health system to minimize selection bias.
Outcome measures
The primary outcome measures in this study are follows: number of ANC attendance, facility delivery (coded yes or no), number of postnatal care attendance and continuum of care. The continuum of care was defined as the percentage of mothers who attended ANC (8 or more), had facility-based delivery, and attended post-natal care within the first 24 hours post-delivery.
Intervention
Our primary exposure of interest is the T4MCH intervention. Technology for maternal and child health (T4MCH) is an intervention introduced to assist in increasing MCH services utilization in some health facilities across Ghana via the delivery of SMS/voice messages to pregnant women. The intervention is novel with the first SMS/Voice message generated and sent to the pregnant women in the 4 health facilities on 1st August 2017.
This intervention was funded and implemented by the Savana Signatures. Savana Signatures is an NGO based in Tamale in the Northern Region of Ghana. They are funded by Global Affairs Canada and the lead partner, Salsan, which has put together a consortium that combines the successful features of T4MCH, with innovations from Mustimuhw Information Solutions.
The intervention works via a designed mobile application known as SGS collect. The app enables health workers to collect personal information of pregnant women attending antenatal services via a smartphone onto a platform called Kpododo. In addition, the app delivers weekly SMS to literate women in English and voice messages for non-literate women in their preferred language to constantly educate and remind them of MCH services at least twice a week. The content of the message includes the need for attending MCH services, the specific date and time to visit health facilities, and general education on self-care and pregnancy risk aversion.
It also has the Knowledge Sharing Sessions (KSS) component where health workers undertake health education on Maternal and Child Health services both at the community level and the facility using audio-visual aids such as laptop computers and projectors as well as a public address system. This also provides a platform for mothers to share ideas with one another on MCH issues.
Health workers within the project area received ICT tools such as mobile phones, laptop computers, projectors, and public address systems. They also received training on the effective use of ICT tools to design, package, and effectively communicate maternal health content to women.
The project was piloted in 2011 and went into its first implementation phase from 2012 to 2014 in larger hospitals in Tamale and some districts in the Northern region. In 2015, it was scaled up to 5 additional smaller facilities in the Northern region, and by 2016 the project beneficiaries had increased to 33 health facilities across the three northern regions of Ghana.
Target population
The intervention targeted all pregnant women who had their first ANC visit in the intervention facilities. Thus, the focus of this study is to evaluate the impact of the intervention on women who did their first ANC visits at the facilities located within the intervention communities.
Covariates
To assess the impact of the intervention on the primary outcome measures, the following variables were adjusted for in all the multivariable regression analyses: age in years, marital status, educational level of mothers, occupation, parity, gestation at registration, and distance to the health facility.
Sampling design
All ANC registrants’ records available for the first 2 months (August 1, 2017, to September 30, 2017) of implementation of the intervention in all the four implementing facilities were reviewed retrospectively from antenatal through to delivery and postnatal. Four other facilities similar to the intervention facilities were purposively sampled in the comparison district. All records of ANC registrants within the same period were reviewed retrospectively from antenatal through delivery to postnatal. In all, 530 ANC registrant’s records were reviewed out of which 469 with complete data (263 study group and 206 comparison group) were used for the study. Data extraction was done using a well-designed data extraction tool. This tool was divided into three parts: demographic history, health facility characteristics, and health facility utilization. The demographic characteristics include age, level of education, marital status, parity, and occupation. The health facility characteristics included proximity to health facility and staffing. On the other hand, the health facility utilization consisted of the number of antenatal care attendance, institutional delivery, and the number of postnatal visits. Data were extracted from antenatal, delivery, postnatal registers, and ANC records booklet of clients with the help of trained research assistants. This study was given a formal waiver for the need for consent by the Ghana Health Service Ethics Review Committee of the Research and Development Division.
Statistical analysis
The data was entered and cleaned via MS Excel and exported to Stata MP version 17 (StataCorp LP, Texas, USA) for analysis. Pearson’s Chi-square and Fisher’s exact test statistics were used to assess the association between antenatal visits, place of delivery, postnatal visits, socio-demographic characteristics, and the intervention. \(T\)-test statistic was also used to assess the mean age at ANC registration between the intervention and the comparison groups. The study estimated average treatment effects (ATEs) of the intervention for the number of antenatal care attendance, skilled delivery, and postnatal services utilization using the doubly robust method (augmented inverse-probability weights (AIPW), and via matching on the propensity score). The choice of these doubly robust impact evaluation techniques emanated from the study design. This is not a before and after study because no baseline data was collected prior to the implementation of the intervention. Because of this limitation, we employed a more rigorous impact evaluation statistical methods such as matching techniques and inverse probability weighting methods that could be used to quantify the impact of health interventions in the absence of baseline data.
In the ATE or the intention to treat (ITT) estimate, all women registering at the intervention facilities were treated as having received the T4MCH intervention, regardless of whether they signed up to receive the intervention or not, that is, we defined as \(ITT=\mathrm{Average}[\mathrm{Y}(\mathrm{i},1)-\mathrm{Y}(\mathrm{i},0)]\) where \(\mathrm{Y}(\mathrm{i},1)\) is the estimate of the outcome measure for beneficiary \(i\) who visited the intervention facilities and \(\mathrm{Y}(\mathrm{i},0)\) is the counterfactual scenario. We did not conduct per-protocol analysis (average treatment effect on the treated) as our primary objective was to determine the average impact of a program on the population targeted by the program.
We conducted balancing diagnostics to determine whether the matching improved covariate similarity index between the intervention and controlled study participants using a kernel density plot that compares the distribution of the propensity scores before and after matching. Since different impact estimation techniques may result in different impact estimates, we conducted a sensitivity analysis to show or prove that even when a different estimation procedure is used, we will still realize a positive impact or otherwise of the proposed intervention. The following sensitivity analyses were conducted: inverse-probability-weighted regression adjustment (IPWRA), 1:1, 1:2, 1:3 nearest neighbour matching, and propensity score matching. We modeled the number of ANC and PNC using the Poisson regression model with robust standard error to account for clustering at the facility level. Since the place of delivery is a binary indicator outcome measure, the logistic regression model was used. The baseline characteristics of program beneficiaries often differ systematically from those of non-beneficiaries, therefore, we accounted for systematic differences in baseline characteristics between T4MCH intervention beneficiaries and non-beneficiaries when estimating the effect of the intervention on outcomes of interest. We adopted matching procedures to reduce the covariate imbalance between the intervention and control groups since our data originate from an observational (or nonrandomized) study design (Fig. 1).
Kenel density of the propensity scores before and after matching for intervention and control populations.
This study emphasizes the use of the size of the impact estimates and not the p values of the estimates based on the recommendation from the American Statistical Association which states that scientific conclusions and business or policy decisions should not be based only on whether a p value or statistical significance passes a specific threshold and a p value, or statistical significance, does not measure the size of an effect or the importance of a result and a p value does not provide a good measure of evidence regarding a model or hypothesis7.
Ethical approval
Ethical clearance to conduct the study was given by the Ghana Health Service Ethics Review Committee of the Research and Development Division before the data collection. The approval reference is GHS-ERC 036/03/19. Permission was also granted by the District Directors of Health Services and Health Facility In-charges of the respective districts where the data was collected. The data reviewed were used solely for this research. We state that informed consent was not applicable in this study as we only analyzed secondary data from patients records that were de-identified. The Ghana Health Service Ethics Review Committee approved the study after carefully examining all the protocols. Permission was as well sought from the Sawla-Tuna-Kalba and Bole district health directorates.