Deciphering the Research on Children and Families
As the funding announcements roll in, there’s a lot of emphasis on “research-based” or “evidence-based programs.” But, like the word “natural” in the grocery store aisle, these terms aren’t standardized and can mean very different things to different people.
Thus, whether you’re writing grants or providing input on programs and funding, it can be challenging to make sense of the research. With a deep commitment to translating research into practical applications that make a difference for families, we suggest the following simple key questions as a guide.
If you have research questions, please consider us a resource. We will be delighted share what we know and direct you to other experts.
How big is the sample?
The larger the number of subjects, the more important the results.
It’s always good practice to look-up the sample size of a study; the larger the number of subjects (“the N”) in a study, the more important the results. Small studies (i.e., fewer than 100 participants) are important and informative, but the findings cannot be generalized or applied to other groups or populations. Whenever you hear a percent quoted (50% of couples experience…) look up the reference and track down the sample size. Becoming Parents Program™ has been or is currently being evaluated with nearly 3500 diverse couples (that’s 7000 parents, not including their children) in this country.
Who are the subjects?
Some studies focus on specific groups, so the findings don’t apply to everyone.
Researchers take a “sample” of people from the population they want to study for a manageable number of participants upon which to conduct research. In larger studies, researchers hope to apply the study’s findings to the whole population from which the sample came. Strong systematic research yields large samples of diverse background within the target population. For example, to understand adult expectant couples, Becoming Parents Program research samples include married or unmarried couples with a range of income or no income, and a range of race/ethnicities, expecting a first or subsequent child. If our study was limited to any one subgroup: only unmarried, white, first time parents, the results would only predict outcomes for that same type of expectant couples.
Is it a randomized-controlled trial?
The gold standard of research is a randomized-controlled study (RCT).
The most rigorous way of determining if a cause-and-effect relationship exists between a program and outcomes is with a randomized controlled study or trial. In RCTs, the sample is recruited from the broad target population (e.g., adult, unmarried expectant couples) and participants are randomly assigned at enrollment (typically by a computer using a mathematical equation) to receive the program (“intervention group”) or not (“control group”). When a sample is large enough, the control group and the program group will be similar in every way except one group got the program and the other didn’t. Thus, researchers can attribute the outcomes of the study to the program as opposed to other factors like sample differences or characteristics. RCTs are often viewed as the gold standard of research when determining efficacy of an intervention, but they cannot be done with every program or product. If a program has appropriately been evaluated with a RCT, especially when the sample size is large, the findings are statistically significant, and the target populations allow for generalizable results to your client population, you can rest assured that the program is strong and is a wise investment. Becoming Parents Program has been or is being evaluated as part of three RCTs in the United States.
Are the results statistically significant?
If so, it is extremely likely that the intervention being tested caused the outcome reported.
When an outcome is said to be significant, in terms of statistics, the outcome was not likely “accidental.” This means it is extremely likely that the intervention being tested caused the outcome being reported. In order to have statistical significance, the sample size must be large enough to have “power” in the evaluation. A research finding may be significant, but not important in “real life”, and conversely, it may be important but not statistically significant. Becoming Parents Progam results have been both statistically significant and important improving relationship quality and satisfaction, reducing depression, improving father involvement, and increasing the success and quality of the co-parenting relationship.
Have the results been replicated?
It proves the intervention has more to do with the approach or program than the people delivering it.
If a study has only been done in one location with a small group, the findings can only be applied to those from the study. Further testing must be done: another person doing the program, in another location, perhaps with a different population, before one can say for sure that the program (rather than the person teaching it) produces the outcomes. When you see a statistic, (50% of couples experience…), look for the number of times the program has been evaluated, where, and by whom before feeling confident that statistic can fairly be used to describe you or me. Becoming Parents Program has been replicated in three sites/studies in different locations with different instructors following the original evaluation.
What about new programs that don’t have research?
While funders often give priority to “proven-programs,” establishing a foundation of evidence and research actually takes years. Both proven and emerging approaches are important to the field.
It is not possible for a program to be both “brand new” and “evidence-based.” There are times when it is important to try new or emerging projects, and times for implementing programs that have been rigorously tested and are effective. The federal Administration for Children and Families has said, for the next round of marriage and fatherhood funding, agencies and programs that have received this ACF funding before and have demonstrated effective results will have priority. Becoming Parents Program was the only program with statistically significant positive impacts in the large-scale ACF-funded study of marriage and fatherhood programs for expectant couples, Building Strong Families.
Becoming Parents Program
Based on 30 years of systematic research and development, the Becoming Parents Program is among the few for expectant and new parent couples with a solid evidence base and broad applicability; and the only program with statistically significant positive impacts in the largest randomized-controlled evaluation of its kind.