AEE 577: Evaluation in Agricultural and Extension Education

NC STATE UNIVERSITY

"Careful analysis of programs and the context in which they operate is a significant precursor to planning any sort of evaluation endeavor."

Wholey, J. S., Hatry, H. P., and Newcomer, K. E. (2004).  

Class V:  Planning Evaluation Process
 

Last week you studied the logic model and focusing evaluation. This is the beginning step of evaluation planning process. The discussion about planning evaluation process will continue in this class. Planning and designing evaluations require difficult decisions. Evaluability assessment is a process that helps evaluators to identify evaluations that might be useful, explore what evaluations would be feasible and design useful evaluations. Evaluation is not always conducted after the fact. In the case of needs assessment, we should conduct the evaluation prior to program development and use that information to guide our planning process. Implementation evaluation has variety of purposes, but looks at implementation and provides feedback on improving implementation processes.

Instructional Objectives:

Upon completion of this lesson, students should be able to:

1.      Describe the key steps in evaluability assessment.

2.      Discuss the four characteristics of an evaluable program.

3.      Describe issues and problems that arise when performing evaluability assessments. Discuss potential solutions to those problems and issues.

4.      Describe implementation evaluation.

5.      Discuss under what circumstance evaluation is unlikely to lead to improved program performance.

6.      Discuss ways in which implementation evaluations are used.

7.      Describe the transparent box paradigm.

8.      Describe the four stages relevant to implementation evaluation.

9.      Discuss the selected implementation evaluation methods for the four stages.

10.    Describe planning steps of evaluation.

11.    Describe evaluation questions.

12.    Discuss what an indicator is for evaluating programs.

13.    Identify resources needed to conduct an evaluation

14.    Draw a timeline for the completion of evaluation steps.

15.    Describe Evaluation Planning Process.

Required Instructional Activities

1. View the Power Point presentation. It provides an overview of the class content.

2. Read "Evaluability Assessment" (pages 33-62) in the course text, Handbook of Practical Program Evaluation by Wholey, Hatry, and Newcomer (2004).

3. Read "Implementation Evaluation" (pages 63-97) in the course text, Handbook of Practical Program Evaluation by Wholey, Hatry, and Newcomer (2004).

4. You will use this Planning a Program Evaluation: Worksheet by Taylor-Powell, Steele, Douglah, to guide you in focusing your evaluation project for the class.  You should start with the first section and begin to work through the section on “Focusing an evaluation” for your major project. This is not to be submitted now, but to get you started thinking about what program you will focus on to develop your evaluation project. 

Suggested Instructional Activities

1. Print off the instructional objectives for this class as a guideline as you work through the readings this week.  You should be able to address each one.

2. Consider how the information provided in these chapters can help you better assess your programs.  How might you use the information gained to better improve program offerings and reach new clientele groups?

Required Assignments

All items, unless otherwise noted, are due by 12:00 noon on Monday following the week of the class.

1. Post your questions or comments on the Web CT Vista discussion board and reply to at least two postings submitted by your classmates.

2. Reading Assignment # 4 from class IV is due by February 9: Open this reading assignment, complete and e-mail as an attachment to the instructor (jay_jayaratne@ncsu.edu). Remember to type your name and save your completed assignment before you e-mail it.

3. You should begin developing questions that your project evaluation will seek to address. Program description of the term project is due no later than February 4.

References

Taylor-Powell, E., Steele, S. & Douglah,M. (1996). Planning a program evaluation worksheet. University of Wisconsin, Cooperative Extension.  Available: http://learningstore.uwex.edu/pdf/G3658-1W.PDF

Wholey, J. S., Hatry, H. P., and Newcomer, K. E. (2004). Handbook of practical program evaluation (2nd edition).  Jossey-Bass, San Francisco, California.