Table of Contents
This program evaluation will focus on teenage pregnancy and Teen Living Programs (TLPs). A TLP is a residential program that offers teen mothers, ages 13-19, support with pregnancy and parenting issues. The rationale for this program evaluation stems from a literature review of current societal challenges teenage mothers face in learning to successfully live independent and productive lives, while also making healthy choices for their children. In response to the literature and the abundance of clients who are in need of support services for themselves and their young children, Better Beginnings, a preexisting TLP, has hired our agency to conduct a program evaluation of their current services to this vulnerable population. This program evaluation will take place over the course of a four month time frame and will utilize a mixed methodological approach.
For this evaluation, we will focus on an outcome evaluation as Better Beginnings has asked if their organization is offering the right combination of services to bring about the outcomes they hope to achieve. Better Beginnings is optimistic that their current program increases the mothers’ financial stability, educational/occupational success, and life and parenting skills, as well as decrease the number of repeat pregnancies and incidences of child abuse/neglect. While the program has these overarching goals, due to time and cost restraints, the program director has asked us to focus our evaluation on efficiency of the services that address issues of life skills and parenting skills.
Better Beginnings is a small, non-profit organization located in Quincy, Massachusetts. Better Beginnings is funded by a combination of money from the government and charitable donations. The program provides support and living services to pregnant teenagers or teenage mothers up to the age of 20. The program facility is a house that holds a maximum of 15 mothers with their children. The services provided include housing, case management, and parenting and life skills classes for mothers and fathers. The clients are required to pay 1/3 of their income for their monthly rent. Additional requirements include the teenager being enrolled in school or working full-time. They also must meet with their assigned case worker on an ongoing basis to discuss progress, concerns, and establish personal goals. The life skills classes include topics such as finances (i.e. balancing checkbook and budgeting), nutrition, parenting, and stress management.
The goal of Better Beginnings is to assist young teenage mothers in transitioning into the community with financial stability and strong personal living and parenting skills. Better Beginnings has four objectives for their program: improve financial stability, decrease the number of incidents of abuse/neglect, enhance community supports, and augment life skills.
The target population for this program is teenagers living in the South Shore in Massachusetts who are lacking family support. Additionally, all of the teenage mothers are DCF involved.
Teenage pregnancy has continued to be of increasing concern to our society as a whole. According to the Centers for Disease Control and Prevention (2010) the overall pregnancy rate for teenagers ages 15-19, in 2002, was 76 pregnancies per 1000 adolescents. Thomas & Looney (2004) estimated that one million teenagers find out they are pregnant every year. Out of those numbers, 95% of the pregnancies are not planned. One of the factors that play a role in child abuse and neglect is when a parent has inappropriate expectations for their child(ren) in addition to a lack of knowledge about child development (Scherma & Korkames-Rowe, 1990). Risks for teenage mothers include lack of education and a high potential of living in poverty, as well as having an increased likelihood of having a second or third child within two years of giving birth (Akinbami, Cheng & Kornfeld, 2001). According to Akinbami et al., (2001), teenage mothers who are of low socioeconomic status are 39% more likely to get pregnant again within the first year of delivery. The percentage increases to 50% within two years of giving birth. Furthermore, each supplementary childbirth raises the probability that a teenage parent and her children will remain living in poverty for an extended amount of time. This is due in fact to the teenagers’ inability to go back to school or obtain a job (Akinbami, Cheng & Kornfeld, 2001).
With the elevated numbers of teenage pregnancy, improving and increasing the education around helpful interventions with regards to this issue in addition to learning ways to cope with teenage pregnancy is vital. This education needs to be provided to teenagers, their parents or guardians, and various mental health and healthcare providers. It’s also important to remember that every teenage mother may have different needs depending on their age, where they live, how many children they have and the ages of those children, their family and support systems, relationship status, level of education, maturity level, and their socio-economic status (Scherma & Korkames-Rowe, 1990). With ample support, a higher number of teenage parents may be able to effectively master parenting and the difficulties and changes that the adolescent years bring. The reality is that teenagers are being sexually active and numbers have shown that many get pregnant. This is a part of our society and it needs to be addressed openly and practically (MacGregor & Newlon, 1987).
Teen Living Programs
Teen Living Programs (TLP) were established in Massachusetts in 1996 following the welfare reform legislation of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) of 1996 (P.L. 104-193) (Collins, Stevens, & Lane, 2000). The Department of Transitional Assistance (DTA) funds TLPs in Massachusetts and the program is overseen by the Department of Children and Families (DCF) (Collins, Lane, & Stevens, 2003). The implementation of the TLP followed the concern that a growing number of teenage mothers were receiving welfare for prolonged periods of time. The anticipation that if teenage mothers were given a safe and supportive environment in which to live there would be a reduction in the number of teen mothers who enrolled on welfare is what propelled the intervention forward (Collins et al., 2000). Additionally, the instatement of TLPs brought about recognition that many teen mothers did not have a safe or supportive home environment. Sadly, many of these individuals were living in homes where there was an increase risk of parental verbal/physical/sexual abuse, neglect, or substance abuse. Therefore, the legislation required teenage mothers who lived under such conditions to instead live in a safe setting consistent with federal legislation. In addition to offering an alternative living situation, TLPs provide skill development curriculum that focuses on parenting skills (Collins et al., 2000).
The outcomes that the federal government intend to achieve by implementing TLPs are the attainment of a high school diploma (or GED); development of parenting skills, which may also lead to a decrease in child abuse and neglect; decrease in subsequent pregnancies; ability to make health choices for the mothers and their children; normal and healthy growth and development of children; and skills to live independently and productively without assistance from welfre (Massachusetts Department of Social Services (MDSS), 1996, as cited in Collins et al., 2000). These outcomes are based on providing a wide array of services to teen mothers. On the other hand, some researchers (Jackson, 1999; Schorr, 1979), argue that teen mothers require a more comprehensive approach, one that focuses more on the integration between employment, post-high school education, and child care.
After conducting a statewide program evaluation, Collins et al. (2003) found that in all 22 Massachusetts TLP sites, all provided the following components: education (high school, GED, vocational, college); employment services; child care resources; health services; life skills/parenting skills instruction; counseling; recreation; case management; transportation; housing search assistance; and follow-up services. Some sites were found to differ in how much emphasis was placed on certain components, but for the most part, the sites were offering all specified components. The researchers also asserted that there were critical issues in implementation regarding programming and staff, standardization, flexibility and individualized services, appropriateness of residential model, and systemic factors (i.e. discrimination) (Collins et al., 2003).
Data that informs program directors about client satisfaction is also necessary to achieve desired outcomes. In a survey conducted by Collins, Lemon, and Street (2000) of 288 teens that were served by TLPs throughout Massachusetts, the researchers found that satisfaction varied from great experiences, poor experiences and, most often, mixed experiences. Components of TLPs that the teens rated as satisfactory were child care and school/GED services. Conversely, the components that the teens rated as unsatisfactory were life-skills classes, housing search services, and employment assistance. Additional data included higher levels of satisfaction in smaller and more intimate programs; a strong sense of group support exhibited by staff members and other residents; a desire for more independence (i.e. less rules and restrictions) for teens living in TLP; and a miscommunication between the program’s intent and the teens’ belief that the program is merely an independent-living program (Collins et al., 2000).
Gaps in the Literature
To date there is limited literature regarding males in TLPs, even though males who are receiving welfare are technically eligible for services (Collins et al., 2003). As stated by Rangarajan and Gleason (1998), the 1996 welfare reform that was written in response to the high rates of teenage pregnancy has encouraged fathers’ participation in order to decrease public subsidy for single mothers as well as other roles unwed fathers have in being responsible parents. However, as suggested by Smith, Buzi and Weinman (2002), in order to make the most of programs for young fathers, “ programs must be able to address needs of young fathers that are based on research and are translated and delivered through a well- planned program” (p.89). Finally, research by Spingarn and DuRant (1996), reports the importance of programs targeting teen fathers because review of studies on young fathers have suggested that fathering a child as a teenager is associated with high risk behavior such as drug abuse, sexually transmitted diseases, violence and school dropout (as cited in Smith, Buzi & Weinman, 2002).
There is also ambiguity around the role of counseling/therapy in TLPs. For example, Collins et al. (2003) found that of the 22 sites evaluated, all included a counseling component. However, most sites reported that there were not sufficient resources available to provide consistent therapeutic opportunities. In addition, offering therapeutic services is not an objective of TLPs, despite concerns from staff members that they feel a therapeutic model of services is appropriate and warranted (Collins et al., 2003).
The hypothesis governing the proposed program evaluation is teenage parents who participate in the Better Beginnings Program will gain the parenting knowledge and life skills necessary to increase their ability to lead independent, healthy, productive lives for themselves and for their children. As such, this program evaluation will focus on research questions specific to the life skills and parenting classes Better Beginnings offers its clients. The evaluation process will assess how well the program develops these skills with the participants, and how successful the program is at helping participants achieve progress toward the desired outcomes. Better Beginnings has set a goal of a 20% increase in participants' knowledge and behavioral/attitude change with regards to parenting and life skills. According to the research literature review regarding evaluations of TLPs, teaching parenting and life skills are critical to the successful outcomes of programs such as Better Beginnings in terms of their ability to help prevent repeat pregnancies; improve the health outcomes for both parents and children; improve overall parenting skills; decrease child abuse and neglect; and increase self-esteem and self-confidence (Akinbami, Cheng, & Kornfeld, 2001).
Conceptualization and Operationalization
The life skills training class at Better Beginnings incorporates food preparation, hygiene, basic literacy, budgeting, preparation for workplace, time management, opening a bank account, attitude, responsibility, communication and problem solving. The class consists of educational instruction, with 12-15 participants attending three, two-hour long classes, weekly, for an eight week period. Therefore, our evaluation will measure the increase in knowledge and change of behaviors and attitudes of participants with regards to each of these topics prior to attending the class and again at the end of the eight-week period.
Parenting skills entails the teens participating in sessions to learn age appropriate activities, discipline, and age appropriate consequences and expectations. The children’s cognitive, emotional and physical development domains will be incorporated in the curriculum along with a nurturing component to improve mother-child relationship. The parenting skills class also consists of educational instruction, with 12-15 participants attending three, two-hour long classes, weekly, for an eight-week period. Therefore, our evaluation will measure the increase in knowledge of participants and change of behaviors and attitudes with regards to each of these topics prior to attending the class and again at the end of the eight-week period. Progress towards self-sufficiency is the optimal goal of this program and is accomplished through the young mothers' increased parenting and life skills as defined above in order to obtain steady employment and consequently exit public assistance (Collins et al., 2003).
This evaluation design approach is a summative outcome and efficiency evaluation to gather data to analyze and interpret whether or not participants in the program have increased their knowledge of parenting and life skills by attending the classes offered by Better Beginnings. Our evaluation design will consist of a mixed methods study that will include both qualitative and quantitative methodologies that will help Better Beginnings determine whether these specific classes are effective and where they can make adjustments too the curriculum to achieve better outcomes.
Specially, the quantitative component will measure the knowledge, skill, and behavior changes reported by the participants after they complete the classes through the use of a pre-test/post-test design. The qualitative component will measure verbal and written responses of participants at the completion of the classes through the use of a focus group study. We will also conduct a focus group with staff members to elicit their observations and perceptions of the teens’ interactions outside of the classroom. Through this mixed methods approach, both sides of the class instruction impact, quantifiable change and participant experience, can be evaluated. This approach also allows for triangulation of data to increase internal validity by using more than one measurement of data (Rubin & Babbie, 2010).
The study sample methodology will consist of a comparison group: the teens that participate in the Better Beginnings program themselves. The evaluation team recognizes, however, that a limitation of this sample plan is that it will not produce results that are generalizable. Participation in the research is voluntary and participants will be informed about how the data will be used and their right to confidentiality. Eligibility requirements for the evaluation would include participation in at least twenty of the twenty-four sessions conducted. To enhance recruitment and retention, participants will be offered an appropriate stipend, as chosen by Better Beginnings, for their time and effort (Babbie & Rubin, 2010).
Data Collection Tools and Plans.
To collect quantitative data, a self-report survey instrument will be administered to participants both before the class and after the class to measure changes. This pre-post test design allows for a baseline of the participants knowledge to be established that can then be compared to a second survey administered at the end of the parenting class to identify whether participants changed or improved their knowledge and skills. The survey will employ a Likert scale and the questions will focus on the curriculum topics specific to the parenting skills class and the life skills class.
In addition, the use of the adult-adolescent parenting inventory (AAPI) will be used to assess the parenting and child-rearing attitudes both pre and post parenting class. This is an important tool that measures the index of risk for parent behaviors known to contribute to the maltreatment of children (CYFERnet, 2010). This tool will also be administered both pre-and post parenting class participation.
To collect qualitative data about the experiences of the participants in the class, a focus group study will be conducted at the end of both the parenting and life skills sessions. Participants will be asked open-ended, verbal questions designed to assess their self-reported feelings of self-esteem, confidence in their parenting skills, and what changes they would make to the classes to make the experience more effective. The focus group study will be in the form of a 30-minute group interview and will consist of all willing participants. Our evaluation team recognizes that the teens come from diverse backgrounds and so every effort will be made to ensure that the evaluations are conducted in a culturally sensitive manner, including offering the survey instruments in both English and Spanish.
Data Processing and Analysis
As this is an evaluation of a relatively small sample size, data collected from the surveys will be tabulated and processed by using Microsoft Excel. This program is well-suited to perform data processing functions including: creating a database of responses, coding of data, cleaning of data, calculating average (mean), range, standard deviation, and creating cross tabulations/pivot tables (Learningstore, 2010). Data gathered from the focus groups will be reviewed and organized into similar categories (e.g., concerns, similar experiences, recommendations, strengths, weaknesses). The evaluation team will them attempt to identify patterns in the themes. Results of both the qualitative and quantitative data analysis will then be interpreted to determine if the findings are consistent with Better Beginnings' goal of a 20% increase in participants' knowledge and behavioral/attitude change with regards to parenting and life skills.
This evaluation will take place over the course of a four-month period. The first phase will involve identifying participants and the collection of data through administration of pre/post surveys and by conducting the focus group. The second and third phase will involve processing and analyzing the data collected. The final phase will be to report our findings and recommendations to Better Beginnings (see Appendix A for a graphical timeline).
The evaluation of the Better Beginnings TLP program suggests new strategies for program development, policy and social work practice. The current data shows the effectiveness of the implemented life and parenting skills trainings in helping the young mothers reduce the levels of abuse and neglect, improve their health outcome, reduce rates of repeat pregnancies among them and increase participation and completion of the GED and high school programs. However, the findings point out some difficulties that these young mothers were having regarding employment and in acquiring safe and affordable priced housing as they were exiting the TLP. These challenges regarding employment and stable housing as reported by Collins et al. (2000), discuss the lack of the welfare reform living requirements in general and TLPs in particular to address the labor market conditions and the supply of housing. Consequently, one of the critical policy responses to address the employment issue affecting those young mothers could be an increased emphasis on trainings for nontraditional jobs that pay higher wages, postsecondary educational support in terms of financial aid and research based interventions with the teen fathers in order to increase their level of commitment and responsibility in caring for their kids. As far as housing, the evaluation team suggests that new policy should address the construction of safe and affordable housing in most communities as recommended by Collins et al. (2000) which could increase the possibility of better outcome results for young mothers exiting TLP programs.
One of the limitations encountered by the evaluation team is the cost restrains presented by the agency which impeded a more detailed assessment of other important components of this TLP program beside the parenting and life skills trainings. Moreover, the limited time provided for the evaluation (four months) was a barrier in facilitating the team interaction and assessment, particularly of those teen mothers who were completing the program.
The current findings will be presented to the board directors and the executive staff and specific recommendations will be suggested regarding the importance of the agency's ongoing attention to implementation issues as well as increased collaboration with other agencies to pressure for suggested policy implementation. The findings will be added to the current literature through a publication in some of the most important social work journals.