AEE 577: Evaluation in Agricultural and Extension Education

NC STATE UNIVERSITY

"The designs of impact evaluations needs to take into account two competing pressures: on the one hand, evaluations should be undertaken with sufficient rigor so that relatively firm conclusions can be reached; on the other hand, practical considerations of time, money, cooperation, and protection of human subjects limit the design options and methodological procedures that can be employed.”

Peter H. Rossi and Howard E. Freeman (1999)

Class VII: Designs of Evaluation Studies
 

This week, you will learn common designs of evaluation studies. These evaluation studies include quasi-experiments, randomized experiments, meta-analysis, systematic review, and research synthesis. As budgets become tighter and debates rage over the payoffs of investments in programs, there is a growing call for evidence of impacts and effectiveness from evaluation data. This class will explore the use of randomized experiments to obtain unbiased estimates of the effectiveness of educational programs. A single study in one area does not mean similar results would be found in other areas. Meta analysis is the statistical synthesis of the data from separate but comparable studies which will provide a quantitative summary of the pooled results, and attempts to move beyond the single evaluation study, information on how and why you might use meta analysis will be presented in this class.

Instructional Objectives:

Upon completion of this lesson, students should be able to:

1.  Describe the purpose of evaluation studies

2.  Explain types of evaluation studies.

3.  Discuss the various threats to internal validity.

4.  Describe the four types of quasi-experimental designs.

5.   Describe quasi-experimentation and compare to randomized experiments.

6.  Describe the concept of randomized experiments and why they are important.

7.  Define systematic review.

8.  Describe meta-analysis.

Note:  Many of the Internet reading assignments require you  to have the Acrobat Reader on your computer. If you don't have this program, it can be downloaded for free from Adobe. (Be sure to select the free version.) Click here to visit the Adobe site.

Required Instructional Activities

1. Read "Quasi-Experimentation" (pages 126-149) in the course text, Handbook of Practical Program Evaluation by Wholey, Hatry, and Newcomer (2004).

2. Read "Using Randomized Experiments" (pages 150-175) in the course text, Handbook of Practical Program Evaluation by Wholey, Hatry, and Newcomer (2004).

3. Read "Meta-Analysis, Systematic Reviews and Research Syntheses" (pages 176-201) in the course text, Handbook of Practical Program Evaluation by Wholey, Hatry, and Newcomer (2004).

4. View the Power Point presentation. It is the summary of class materials.

5. Read Taylor's thoughts on  Data gathering designs to evaluate educational programs. 

Suggested Instructional Activities

1. Read Diem's article "Using research methods to evaluate your extension program." 

Required Assignments

All items, unless otherwise noted, are due by 12:00 noon on Monday following the week of the class.

1. Post your questions or comments on the Web CT Vista discussion board and reply to at least two postings submitted by your classmates.

2. Since you have the mid-term exam, there is no reading assignment for this week.

3. Open this mid-term exam, answer all questions and e-mail as an attachment to the instructor (jay_jayaratne@ncsu.edu). Remember to type your name and save your completed exam before you e-mail it. Completed midterm exam is due by February 25.
4. Evaluation plan of your term project is due no later than February 18. Please refer evaluatin plan guidelines in the course information.

References