Exemplary
Program
Evaluations
 |
|||||
| State: | Missouri | ||||
| Project Name: | Skills Development through Community Computing: The Irondale Community Computer Lab | ||||
| Targeted Group : | Community (school-age children, teens, adults and seniors | ||||
| Number involved: | 54 (18 children, 7 youth, and 29 adults) | ||||
| Activities provided: | Imparting computer skills through hands-on use, informal tutoring from volunteer technical staff, interaction with other computer users, and on-line training. | ||||
| Where: | At ICCL (Irondale Community Computer Lab) in Irondale City Hall, Irondale, Missouri | ||||
| When: | Participants averaged 21 visits to the lab or 1.6 visits per week. The lab was open for 13 weeks during the quarter. | ||||
| Who: | Volunteers from Irondale and Big River Community Information Network, along with University Outreach and Extension staff, provide tutoring and guidance. Five individuals volunteered a total of 88 hours to supervise the lab. | ||||
 |
||||
 |
One of the anticipated impacts of the ICCL is an improvement in participants' computer skills by providing the residents with access to computers, access to the Internet, and assistance with information technology. | |||
|
In addition to improvements in participants' computer skills, other anticipated impacts of the program include an increase in after-school adult supervision for children and youth, increased exposure to educational resources and more learning time for children and youth, increased knowledge gains for community members from web-based resources, and increased interaction among community residents. | |||
 |
||||||||||||||
| Methods & Tools | ||||||||||||||
|
To measure the changes in computer skills over time, a skills assessment test was developed. The initial, or baseline, assessment was in December 2001, the second round of assessment was in June 2002, third round of assessment in February 2003, and a last round of assessment in November 2003. The results of the fourth round of assessment have not yet been tabulated. The computer skills assessment instrument is a task-based assessment, meaning that the person being evaluated must complete a series of multi-step tasks to demonstrate competence in each skill area. A trained evaluator conducts the assessment by reading the tasks to the person taking the test and recording whether or not the person is able to complete each of the 96 distinct tasks. |
||||||||||||||
|
The
instrument is designed to measure skills in four common areas:
|
||||||||||||||
|
|
General operation of the computer, | |||||||||||||
|
|
Word processing, | |||||||||||||
|
|
Using the World Wide Web, and | |||||||||||||
|
|
Electronic mail. | |||||||||||||
|
The
assessment starts with very basic tasks, such as turning on the
computer and opening a file, and progresses to more difficult tasks.
It is organized into six sections: 1) basic computer and operating
system, 2) World Wide Web, 3) electronic mail, 4) basic word processing,
5) advanced word processing, and 6) advanced operating system. Word
processing skills are tested using Microsoft Word.
|
||||||||||||||
| Analysis | ||||||||||||||
| Average scores from the first test were compared to average scores on the second test using t-tests. | ||||||||||||||
| Results | ||||||||||||||
| Assessment results looked at: | ||||||||||||||
| 1. Skill levels of lab participants, and | ||||||||||||||
| 2. Changes in skill levels | ||||||||||||||
|
Skill Levels of Lab Participants |
||||||||||||||
| The
overall skill levels of lab participants are indicated by the scores
for everyone who took the test. Since lab attendance was increasing
over time, there were 14 participants who took the test in December
2001, 42 participants who took the test in June 2002, and 54 participants
who took the test in February 2003.
The youth at ICCL received the highest overall average score (79 percent) as well as the highest scores in each skill area (refer to tables in provided reports). Similarly, adults have consistently higher average scores than children in every skill area. All age groups at ICCL have now mastered basic knowledge about computer operating systems and how to use the World Wide Web. Skills in e-mail use are also high, with the average score being 93 percent. |
||||||||||||||
|
Changes
in Skill Levels Over Time
|
||||||||||||||
| Scores
of the 42 people completed the skills assessment in both June 2002
and February 2003 were compared across the two rounds to provide information
on the trends in computer skills over time. All age groups experienced
large and statistically significant improvements in their computer
skills between June 2002 and February 2003. For the group as a whole,
average scores improved by 25 percent. Children had the largest increase
in average scores, with a 43 percent improvement.
The June 2002 report was based on scores from 14 people (six children, three youth, and five adults) who took the test twice, and had used the computer lab for at least 6 months. The results indicate that lab participants improved their computer skills over the six-month period between December and June. The average score for the group as a whole increased from 54 to 72 (p<.01). The improvements in scores were particularly pronounced among children and youth. The impacts were larger and the statistical significance of the results was stronger for the youngest participants. Children experienced the largest increase in skills, with a doubling in their average scores from 24 to 50 (p<.01). Average scores for youth increased from 65 to 80 (p<.10), which represents a 23 percent increase over the base score of 65 percent. Adults' average scores increased by about 11 percent, from 84 to 93 percent, but this increase was only marginally significant in a statistical sense (p=.12). There was also a significant increase over time in participation intensity. Even without formal on-site training programs, the ICCL is having a positive impact on the people who come to the lab, based on improvements in their test scores. The Irondale Community Access Laboratory is a fun place for kids and other members of the community to drop in during the day and after school. There is a sense of pride and ownership with the users of the lab. The atmosphere is safe and cozy. The kids enjoyed the variety of snacks. The kids used the Internet and a variety of software. Recommendations: Use the categories of SACERS to select software to rotate in and out of the lab. The interaction between the kids and the staff was tremendous. The staff facilitated the kids being successful using the computer without being intrusive. Overall, this was a very good and enriching environment for kids to be after school. Evaluation
reports summarizing the results are available through the following
link: |
||||||||||||||
 |
||||
|
To assess program effectiveness | |||
|
Program planning | |||
|
Program modification | |||
|
Documentation in grant proposal writing | |||
|
Reports to collaborators and funders | |||
|
Marketing to the community | |||
|
For support to replicate program in other places | |||
 |
|||
| Dr.
Elizabeth Dunn 9627 SW 40th Lane Gainesville, FL 32608 Phone: (352) 333-0089 Fax: (352) 331-4489 (call ahead to send fax) |
|||