Evaluating the National Outcomes: Children

Program Outcomes for Children

PERSONAL AND SOCIAL ADJUSTMENT

Outcome Component 3: Gets Along With Other People

In our model of personal and social adjustment in children (Ysseldyke & Thurlow, 1993), this outcome component refers to the ability to utilize skills that promote effective interaction with others. Research domains that fall under this umbrella would include studies of Social Competence, Social Skills, Social Problem-Solving, Peer Relations, and Social Information-Processing. Social competence is often used as an umbrella term that encompasses learned social skills as well as related cognitive and emotional capabilities (Hughes & Sullivan, 1988), and includes functioning in structured settings with adults as well as peer relationships (Ladd, 1990). Social skills are the behaviors that contribute to social competence (Gresham & Elliot, 1984; Gresham & Reschly,1987). Social problem-solving is a particular form of social skill that involves the strategies the child uses to appraise a social situation, identify their own feelings and those of others, generate solutions, understand consequences of each, and make choices (Marshall, Temple, Montes, & Russell, 1996). Children who are at risk are often socially unskilled and have difficulty understanding how their actions affect the reactions they receive from others (Rathjen, 1984).

Social competence involves social information processing skills, including perspective-taking (Crick & Dodge, 1994). Emotions also play a role in social perceptions and responses. Children who have been maltreated often show distortion in their social perceptions, and respond inappropriately to peers (Zahn-Waxler & Radke-Yarrow, 1990). Well developed social skills prepare children for interpersonal relationships, improve their ability to resolve life situations, and help them to make healthy choices (Leffert, Benson, & Roehlkepartain, 1997). At the same time, close relationships such as families and friendships provide the settings children need to learn and practice those skills (Hartup & Laursen, 1991).

Because skill in interacting effectively with others cuts across so many aspects of a child's life, children who lack social skills or the cognitive and emotional competencies that underlie them are at risk for poor adjustment in school, family and peer relationships. Such interpersonal skill deficits have been associated in some studies with later delinquency, school drop-out, and substance abuse, and poor self-concept (Bukowski & Hoza, 1989), yet educators report that increasing numbers of children are entering kindergarten with inadequate social skills and behavioral controls (Marshall, Temple, Montes, & Russell, 1996). Understanding this, programs in many communities take a preventive approach by attempting to teach interpersonal and social problem-solving skills to preschool and school-age children. State Strengthening projects and traditional 4-H activities often seek to improve outcomes in this area, whether they use a formal skills curriculum or simply provide opportunities for children to interact effectively with peers or adults.

Measurement of peer relationships can be complex. One of the most common methods is sociometric studies, which ask children in a class or group to nominate or rate their peers in terms of friendship or likability. It may be important to decide whether the outcome of interest is 1) reciprocal, positive feelings between a child and at least one peer, 2) the number of friendships, or 3) the quality of friendships (Bukowski & Hoza, 1989). Another approach, which may be more practical for evaluation purposes, is to use behavioral checklists which are completed by teachers or program staff based on their observations of children's interactions.

The following are some appropriate indicators of positive outcomes for children in the area of getting along with other people, based on the NCEO model (Ysseldyke & Thurlow, 1993), as adapted for community-based programs by the Children’s Outcome Workgroup. The appropriateness of measuring any given indicator on the list for you program evaluation depends on the age of the children you serve, the setting, and the goals and activities of your particular program. These indicators are typical of the kinds of items included in many of the behavioral checklists reviewed in the Measures section below.

Percent of children who initiate and respond to social contacts with other children
Percent of children who engage in extended social interactions with other children
Percent of children who engage in appropriate play with other children (e.g., parallel, associative, and emerging cooperative play skills)
Percent of children who initiate and respond to social contacts with adults
Percent of children who appropriately express needs to other children and adults

Percent of children who appropriately express needs to other children and adults
Percent of children who interact appropriately with other children
Percent of children who have friends own age and are part of a social network
Percent of children who interact appropriately with adults (eg., cooperation)
Percent of children who engage in productive group work (especially older elementary school children)
Percent of children who demonstrate skill in interaction and making decisions in social situations, including during interpersonal conflict (especially older elementary school children)

While there is wide-spread agreement that promoting competence in interpersonal relationships is important, measuring it reliably in children has been more elusive. Some aspects of social competence and peer relationships require fairly subjective judgements, and may vary from one social context to another. Social skills are easier to measure than some of the other aspects of social competence, because they are defined behaviorally. Multi method assessment, using a combination of behavioral rating scales with more qualitative or program-specific assessments, is probably the method of choice for evaluation of social competence programs for children (Gresham & Elliot, 1984; Merrell, 1993; Hightower, Work, Cowen, Lotyczewski, Spinell, Guare, & Rohrbeck, 1986).

Bukowski, W. M., & Hoza, B. (1989). Popularity and friendship: Issues in theory, measurement and outcome. In T. J. Berndt & G. W. Ladd (Eds.), Peer relationships in child development (pp. 15-45). New York: John Wiley.

Crick, N. R., & Dodge, K. A. (1994). A review and reformulation of social information-processing mechanisms in children's social adjustment. Psychological Bulletin, 115(1), 74-101.

Gresham, F. M., & Elliot, S. N. (1984). Assessment and classification of children's social skills: A review of methods and issues. School Psychology Review, 13(3), 292-301.

Gresham, F. M., & Reschly, D. J. (1987). Dimensions of social competence: Method factors in the assessment of adaptive behavior, social skills, and peer acceptance. Journal of School Psychology, 25, 367-381.

Hartup, W. W., & Laursen, B. (1991). Relationships as developmental contexts. In R. Cohen & A. W. Siegel (Eds.), Context and development (pp. 253-279). Hillsdale, NJ: Lawrence Erlbaum.

Hightower, A. D., Work, W. C., Cowen, E. L., Lotyczewski, B. S., Spinell, A.P., Guare, J. C., & Rohrbeck, C. A. (1986). The Teacher-Child Rating Scale: A brief objective measure of elementary children's school problem behaviors and competencies. School Psychology Review, 15, 393-409.

Hughes, J. N., & Sullivan, K. A. (1988). Outcome assessment in social skills training with children. Journal of School Psychology, 26, 167-183.

Ladd, G. W. (1990). Having friends, keeping friends, making friends, and being liked by peers in the classroom: Predictors of children's early school adjustment? Child Development, 61, 1081-1100.

LaFreniere, P.J., & Dumas, J. E. (1996). Social competence and behavior evaluation in children ages 3 to 6 years: The Short Form (SCBE-30). Psychological Assessment, 8, 369-377.

Leffert, N., Benson, P. L., & Roehlkepartain, J. L. (1997). Starting out right: Developmental assets for children. Minneapolis, MN: Search Institute.
Marshall, H. M., Temple, M., Montes, G., & Russell, R. (1996). Enhancing young children's social competence: ENHANCE!/SOCIAL COMPETENCE PROGRAM (ESCP) - A field developed program for children, teachers, and parents. Poster presentation, Head Start's Third National Research Conference, Washington, DC; June 20-23, 1996.

Merrell, K. W. (1993). Using behavioral rating scales to assess social skills and anti-social behavior in school settings: Development of the School Social Behavior Scales. School Psychology Review, 22(1), 115-133.

Rathjen, D. P. (1984). Social skills training for children: Innovations and consumer guidelines. School Psychology Review, 13(3), 302-310.

Ysseldyke, J. E., & Thurlow, M. (1993, October). Developing a model of educational outcomes (NCEO Report No. 1). Minneapolis, MN: University of Minnesota, College of Education, National Center on Educational Outcomes.

Zahn-Waxler, C., & Radke-Yarrow, M. (1990). The origins of empathic concern. Motivation and Emotion, 14(2), 107-130.

The following standardized assessments are provided as examples of measures that may be useful for evaluation of community-based programs. This listing is not comprehensive, and is not intended as an endorsement of any particular measure. Some of the assessment instruments that follow are copyrighted, and require specific levels of training to administer. Prices of some measures may be subject to change. While subscales that appear most relevant for this outcome area are highlighted, evaluators who are considering any standardized instrument will want to individually review the items on each measure and subscale to ensure that they are appropriate for a particular program.

It is important to recognise that standardized measures, such as the ones listed below, are not the only appropriate ways to assess outcomes for children. In fact, it is highly recommended that standardized measures be used in conjuction with other methods, including qualitative interviews and observational assessments, and other indicators which may be obtainable from existing records for some school-aged programs [see sections on Qualitative Interviewing and Using Existing Data elsewhere on the CYFAR Evaluation website]. In some instances, child ratings of their peers (sociometric studies) may be relevant. Other appropriate group-level indicators may include (but are not limited to) such data as school grades, attendance, changes in rates of referral for disciplinary action or family counseling, or rates of participation in extracurricular activities.

1. Child Behavior Checklist (CBCL). T. M. Achenbach and C. S. Edelbrock
Date: 1981, 1988

Ages 2 1/2 to 6 years

Description/Comments:

COR is a teacher checklist developed by the High Scope Educational Research Foundation for assessment of preschoolers. This measure has become the assessment of choice for some evaluators of community-based preschool prevention programs because it is behaviorally focused, can be used in parts (subscales), and is not excessively time-consuming for teachers or program staff to administer. The COR has been successfully used with low-income and ethnic minority populations.

Children 8 years and older (3rd through 9th Grade)

Description/Comments:

Well-validated and widely used self-report measure of one aspect of self-esteem or self-concept (children's perceptions of their competence in several specific age-appropriate domains). A "structured alternative" question format is designed to minimize social desirability response tendencies. It is expected that through the elementary school years, children's self-perceptions in these areas will become increasingly realistic. In program evaluations, Harter suggests that a goal should be to foster a realistic sense of competence rather than just an increase in self-esteem or perceived competence. However, since it is impossible to judge how realistic the child's self-assessment is without some basis for comparison, some evaluators recommend using the teacher-report format of these scales instead of (or in addition to) the child self-report format.

4. Pictorial Scale of Perceived Competence & Social Acceptance for Young Children (PSPCSA) S. Harter & R. Pike
Date: 1984

Preschool and primary-age children (two versions)

Description/Comments:

A picture-based assessment administered in individual interviews with children in either home or school/program setting. A downward extension of the Perceived Competence Scale for Children, this measure is widely used as a reliable, valid, and theoretically sound measure of social self-perceptions of preschool and early elementary children. However, the original psychometric work was based on a small sample of middle-class white children. Subsequent work with low-income samples of younger (preschool-aged) Hispanic and African-American children has not fully supported the reliability and factor structures of the scales for use with these populations. Modified versions for these populations have been piloted (for further information, see Taylor, Callor, Machida, & Marx, 1995). Also, because it is impossible to judge how realistic the child's self-assessment is without some basis for comparison, some evaluators recommend using the teacher-report format of these scales instead of (or in addition to) the child self-report format.

5. Social Competence and Behavior Evaluation Scale, Preschool Edition (SCBE)
P.J. LaFreniere & J.E. Dumas
Date: 1996

Basic Scales are: Depressive/Joyful, Anxious/Secure, Angry/Tolerant, Isolated-Integrated, Aggressive/Calm, Egotistical/Prosocial, Oppositional/Cooperative, and Dependent/Autonomous
Summary Scales are: Social Competence, Externalizing Problems, Internalizing Problems, and General Adaptation

Available Through:

Western Psychological Services
12031 Wilshire Blvd.
Los Angeles, CA 90025-1251
Phone: 1-800-648-8857 (Credit card orders)
FAX: 310-478-7838

Cost:

$75 for kit (includes 25 forms and manual)

Target Audience:

Ages 2 1/2 to 6 years

Description/Comments:

An 80-item scale administered by teachers; takes about 15 minutes per child. Has been successfully used as an outcome measure in evaluation of intervention programs, as well as for research and diagnostic purposes. Although new, endorsements from staff in Head Start, clinical, and other community-based settings suggest that the measure may be appropriate for a variety of at-risk populations. Scales showed high interrater and test-retest reliability, internal consistency, and temporal stability in U.S. and Quebec samples (La Freniere & Dumas, 1996).

6. Social Skills Rating System (SSRS) F. M. Gresham & S. N. Elliot
Date: 1990

Three scales are included. Social Skills includes five positive behavior subscales of Cooperation, Assertion, Responsibility, Empathy, and Self-Control. Problem Behaviors includes Externalizing Problems, Internalizing Problems, and Hyperactivity subscales. Academic Competence is a teacher rating of reading and math performance, general cognitive functioning, motivation, and parental support.

Available Through:

American Guidance Service (AGS)
4201 Woodland Road
P.O. Box 99
Circle Pines, MN 55014-1796
Phone: 1-800-328-2560

Cost:

Starter sets, with a manual and enough forms for 10 students, cost about $97.95 for the Preschool/Elementary Level (check with AGS for current prices). Additional forms cost approximately 60-65 cents apiece. Software is available in several formats to assist in computer scoring and reporting.

Target Audience:

Children ages 3-18. Two versions address three developmental levels: Preschool/Elementary, and Secondary. Grades 3-12 can use self-report forms.

Description/Comments:

Rates social behaviors that are believed to affect areas such as teacher-student relationships, peer acceptance, and academic performance. Behaviors are rated according to perceived frequency and importance. An advantage is that several raters (teachers, parents and children if in Grade 3 or higher) can provide a rounded picture of behavior. Administration takes about 10-25 minutes per questionnaire. Standardized on national sample of over 4,000. Separate norms are provided for boys and girls (ages 3-18), and for elementary students with and without disabilities. Although primarily designed for classroom use, evaluators and staff report that most items seem suitable for community-based programs, including State Strengthening projects. The SSRS is intended to be administered by individuals with graduate training in testing and measurement. AGS will request information on user qualifications when the order is placed. The SSRS uses teacher, parent, and (for children in 3rd grade or older) self-report student questionnaires, with the three perspectives combined on one form. In some program settings with low adult literacy levels, evaluators and staff have chosen to administer the parent questionnaires orally.

7. Teacher-Child Rating Scale (T-CRS) Primary Mental Health Project
Date: 1993 (1986)

The T-CRS consists of two parts, one dealing with classroom problem areas, and one dealing with competency areas. The subscales are empirically derived.
Problem Area Subscales:
     Acting Out Behaviors
     Shy-Anxious Behaviors
     Learning Skills
Competency Subscales:
     Frustration Tolerance
     Assertive Social Skills
     Task Orientation
      Peer Social Skills

Available Through:

Primary Mental Health Project, Inc.
685 South Ave.
Rochester, NY 14620-1345
Phone: 716-262-2920
FAX: 716-262-4761

Cost:

30 cents/form. For an additional 50 cents/form, forms can be computer scored at PMHP, which will provide summaries and individual profiles. Students using the T-CRS for their own research (theses and dissertations) can request permission to reproduce the measure free of charge.

Target Audience:

Primarily used with kindergarten through third grade children, but has been successfully used with preschoolers through high school aged children, with minor adaptation of some items (for example "Learning academic skills" in preschool settings would be interpreted as age-appropriate school-readiness skills).

Description/Comments:

The T-CRS is a 38-item behaviorally-oriented checklist completed by teachers or program staff. Tends to be well-accepted by teachers and program staff because it is quick (3-5 minutes per child), and assesses relative strengths and weaknesses. T-CRS may be used either as a quick screening tool to assess children's emotional, behavioral, social, and general school adaptation, or as a pretest-posttest measure to evaluate progress. It may be used at either the individual or the group level for evaluation purposes. Although developed for evaluation of social skills intervention programs in school settings, the T-CRS has been used in other community-based group settings for children in this age range. Separate norms are available for males and females, and for urban and non-urban residents.

8. Vineland Adaptive Behavior Scales-Classroom Edition: Socialization Domain (VABS)
S. Sparrow, D. Balla, & D. Cicchetti
Date: 1985

Four domains and eleven subdomains:
1) Communication (Receptive, Expressive, and Written)
2) Daily Living Skills (Personal, Domestic, and Community)
3) Socialization (Interpersonal Relationships, Play and Leisure Time, Coping Skills)
4) Motor Skills (Gross and Fine).

Available Through:

American Guidance Service (AGS)
4201 Woodland Road
P.O. Box 99
Circle Pines, MN 55014-1796
Phone: 1-800-328-2560

Cost:

Check with AGS for current prices; prices vary for different versions. Starter sets include manual, 10 questionnaire forms, and parent report forms.

Target Audience:

3-12 years for Classroom edition; birth to 18 years for Interview edition

Description/Comments:

The Classroom edition is a 244-item questionnaire completed by teachers, and takes about 20 minutes to administer. The Interview edition is administered to parents or caregivers as a semi-structured interview, and takes 20-60 minutes. Qualified professionals must interpret the scores. Computer programs may be purchased for analysis, and materials are available in Spanish. Standardized on a large national sample that reflects U.S. census data. Appears to be valid and flexible for use with a wide range of ages.