Program Outcomes for Children

SATISFACTION OUTCOMES

Introduction

Satisfaction surveys are a deceptively simple form of program evaluation.  In the past, participant satisfaction surveys were sometimes the only form of evaluation conducted by small programs.  While it is not sufficient to measure satisfaction as the only outcome of a program, satisfaction is an important starting point, because if participants are unhappy with the service provided, they will not continue to participate, and therefore they will not be able to benefit from what the program has to offer.  For programs serving children, the satisfaction of parents or guardians, the morale of staff members, and the perceptions of members of the community who fund and support the program can be as important as the satisfaction of the children themselves (Ysseldyke & Thurlow, 1993). 

NCEO Satisfaction Model

However, it is not enough for an evaluation to simply ask participants whether they like the program.  People may like or dislike a program, service, or product for very subjective reasons that have nothing to do with how effective it is in achieving its desired outcomes.  For example, children in an educational program that offers little challenge might say that they like it because it makes few demands on them and serves good snacks, but we would not necessarily expect such a program to result in better academic skills for the children. 

Researchers in the fields of marketing and consumer science increasingly realize that satisfaction is a complex interaction of present and past experiences, expectations, and perceptions.  Besides the quality of the product or service, perceptions of satisfaction can be influenced by cost, participant needs, expectations, availability of competing services, and recommendations of family and friends (Oliver, 1997; Anton & Perkins, 1997).  In the business world, customer satisfaction is “a state of mind that a customer has about a company when their expectations have been met or exceeded over the lifetime of the product or service” (Anton & Perkins, 1997, p. 3).  Although the nature of services or “products” is different in community-based programs for children, youth and families, in both cases it is important to ask the right questions, and find out what particular aspects of the program are meeting expectations or need improvement.  Global satisfaction questions (“Do you like this program?”) are usually less helpful than those that are more specific (“Are staff members available to answer your questions?  Do they seem to have the skills and training to help your child?  Do the hours of operation match the times you need services?”).

In the five-tiered model of program evaluation which underlies the State Strengthening Evaluation Guide (Jacobs, 1988; Callor, Betts, Carter, & Marczak, 1997), participant satisfaction is most relevant to the activities of Tier 3, Understanding and Refining Programs.  Assessing participant satisfaction at this stage, through surveys or interviews, can help to determine whether the program is being implemented in the ways it was intended, and whether adjustments and fine-tuning are needed which would improve the effectiveness of the program.  Often fairly 
simple changes can improve the fit between the program services and the needs of participants.
 

Outcome Components

The NCEO model, as adapted for community-based settings, suggests three desirable outcomes in the domain of satisfaction with the program:
 

  • Parent/guardian satisfaction with the program services that children receive
  • Community satisfaction with the program services that children receive
  • Child satisfaction with the program experience
Suggested Indicators

The following are some appropriate indicators of positive outcomes for children in the area of Satisfaction, based on the NCEO model (Ysseldyke & Thurlow, 1993), as adapted for community-based programs by the Children’s Outcome Workgroup.  The appropriateness of any given indicator for your program evaluation depends on the ages of the children you serve, the setting, and the goals and activities of your particular program.
 

  • Percent of parents/guardians who understand program services and rate them as effective, efficient, coordinated, and responsive in meeting their child’s needs
  • Percent of parents/guardians who understand program services and rate them as effective, efficient, coordinated, and responsive in meeting their family’s needs 
  • Percent of parents/guardians who are satisfied with their own level of involvement in program decision making
  • Percent of program staff who understand program and rate it as effective, efficient, coordinated, and responsive in meeting child and family needs
  • Percent of program staff who are satisfied with their own level of involvement with program decision making and delivery of services
  • Percent of community stakeholders who understand the program and rate it as effective, efficient, coordinated, and responsive in meeting child and family needs
  • Percent of community stakeholders who are satisfied with their own level of involvement with program decision making and delivery of services
  • Percent of children who enjoy their participation in the program setting
Summary

Assessing participant satisfaction is a necessary but not sufficient part of evaluating community-based programs such as State Strengthening projects.  Surveys or interviews with parents, guardians, staff, children, and other stakeholders are most useful if they are brief, tailored to your specific program, and focused on aspects of the program that are in your power to change or improve. 
 

References

 Anton, J., & Perkins, D. (1997).  Listening to the voice of the customer: 16 steps to a successful customer satisfaction measurement program.  New York: The Customer Service Group.

 Callor, S., Betts, S., Carter, R., & Marczak, M. (1997).  State Strengthening Evaluation Guide.  Tucson, AZ: University of Arizona, Institute for Children, Youth & Families and USDA/CSREES.

 Jacobs, F. H. (1988).  The five-tiered approach to evaluation: Context and implementation.  In. H. B. Weiss & F. H. Jacobs (Eds.), Evaluating family programs (pp. 37-71). New York: Aldine DeGruyter.

 Oliver, R. L. (1997).  Satisfaction: A behavioral perspective on the consumer. New York: McGraw-Hill.

 Ysseldyke, J. E., & Thurlow, M. (1993, October).  Developing a model of educational outcomes (NCEO Report No. 1).  Minneapolis, MN: University of Minnesota, College of Education, National Center on Educational Outcomes.


 

Measures


Bibliography


| Program Outcomes for Children |

| NOWG Home |