What is Program Evaluation?

First Session:
To define and understand “What is program evaluation?”
To understand the historical foundations of program evaluation.
To identify and develop appropriate evaluation assessment techniques used in educational and other program settings.

"Program evaluation is the systematic collection of information for use to improve effectiveness and make decisions with regard to what those programs are doing and affecting." University of Minnesota http://www.evaluation.umn.edu/

"Essentially you are trying to answer the question, "Does the program do what it says it does?". Because evaluation is on-going your evaluation may steer your client in a particular direction and it will also be used to inform the next evaluation" (Jay Wilson, 2009)

I found the following useful links:
http://www.epa.gov/evaluate/whatis.htm
In its broadest definition, Program Evaluation is a systematic way to learn from past experience by assessing how well a program is working.
- Program evaluation is almost always retrospective, i.e., examining and learning from experience, though it may include prospective elements. For example, an analytical study that makes use of data on past performance to estimate future results would be an evaluation, but one done prospectively to estimate the effectiveness of a new environmental program based on assumptions about its design and/or operation would not be.
- An evaluation can be systematic without being elaborate or expensive. It’s possible to keep it simple and affordable

http://www.epa.gov/evaluate/whatis.pdf

http://www.ocde.k12.ca.us/downloads/assessment/WHAT_IS_Program_Evaluation.pdf

http://www.en.wikipedia.org/wiki/Program_evaluation
Program evaluation is a systematic method for collecting, analyzing, and using information to answer basic questions about projects, policies and programs. Program evaluation is used in the public and private sector and is taught in numerous universities.. Program evaluations can involve quantitative methods of social research or qualitative methods or both. People who do program evaluation come from many different backgrounds: sociology, psychology, economics, social work.

http://gsociology.icaap.org/methods/evaluationbeginnersguide.pdf

http://non-profit-governance.suite101.com/article.cfm/board_member_selfassessment
umass.edu/oapa/oapa/publications/online_handbooks/program_based.pdf
http://www.macleans.ca/education/universities/article.jsp?content=20070323_155000_816
http://www.medicine.usask.ca/pt/general-information/school-of-physical-therapy-operations-manual-1/Program%20Evaluation.pdf/view
http://www.evaluationcanada.ca/site.cgi?s=4&ss=2&_lang=an
http://www.uwex.edu/ces/pdande/evaluation/index.html

Requested Readings:
http://www.managementhelp.org/evaluatn/fnl_eval.htm
http://pathwayscourses.samhsa.gov/eval101/eval101_toc.htm
http://jan.ucc.nau.edu/edtech/etc667/proposal/evaluation/summative_vs._formative.htm
http://delicious.com/wi11y0/809
http://www.evaluationcanada.ca/site.cgi?s=1
http://www.eval.org/

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R.(2004). Program evaluation: Alternative approaches and practical guidelines. White Plains, NY: Longman.
Owen, J. M., & Rogers, P. J. (1999). Program evaluation: Forms and approaches. Thousand Oaks, CA: Sage.
Posovac, E., & Carey, R. (2003). Program Evaluation – Methods and Case Studies. (6th edition). New Jersey: Prentice Hall

First assignment: a description of how to do program evaluation in Canada
http://www.spcottawa.on.ca/CBRNO_website/How2program_evaluation.htm#Client
http://www.mhcbe.ab.ca/cec/specialeducation-studentservices/ECSPROGRAMMINGFORCHILDRENWITHSEVEREDISABILITIES.pdf

PENDING ASSIGNMENTS
Assignment # 5 Design and test a short survey. Include a variety of question types such as scale rating, short answer, and open-ended. Original version and the modified version based on the testing of the survey with four individuals. Deadline: November 20th

Major Assignment: Project
Evaluation Plan (Proposal) Dec. 11 - 50 marks
A Proposed Evaluation of the “Spanish Intermediate & Conversational and Ultra Play - Tennis After School” Programs in a Community Center: A Case Study of the City of Ottawa: Parks & Recreation Master Plan Experience
By Nelson Dordelly-Rosales
The purpose of this paper will be to design an evaluation plan after completion the “Ultra Play - After School” Program in Sandy Hill Community Center: A Case Study of the City of Ottawa – Parks & Recreation Experience. The plan will be a theoretical paper that outlines the program and the goals or objectives to be evaluated. It will demonstrate my ability to analyze a program, integrate the different tools and theories addressed recently into an evaluation plan, determine a suitable evaluation plan and create the instruments I would use to conduct the analysis. Essentially the purpose of this evaluation plan is to convince Mr. Martin Travis, Coordinator of Parks and Recreation that I should be “the evaluator for the evaluation.” Hence, I want to convince him that I am the “best” to perform the evaluation of the above mentioned program, and that I have the best team to help me on this matter.
Through case study, this paper will lend insight to ways through a program-based evaluation, or a logic “improvement-focused” model (Posavac, et. al, 2003) can facilitate a “holistic” approach to the evaluation of ‘Ultra Play-After School program.’ So, an important piece of this evaluation plan is to describe, or elaborate upon, different reasons for selecting this particular model and approach. While the term “program” is used, I find a logic model equally useful for describing group work, team work, community-based collaborative and other complex organizational processes as I seek to promote results-based performance. For presentation of paper, I will use a case study format that includes following components:
• Abstract – a brief summary of the major points of the study as well as a short list of key words.
• Introduction
Prior evaluations or policies in a Community in Ottawa:
http://www.gnag.ca/index.php
http://www.gnag.ca/index.php?page=67&id=20

In comparison to:
http://www.topeka.org/parksrec/garfield.shtml

http://www.uml.edu/centers/CFWC/Community_Tips/Program_Evaluation/Program_Evaluation.html

http://www.skaneatelescommunitycenter.com/index.php?option=com_facileforms&Itemid=124


A case in Florida Miami Beach:
http://www.shakealegmiami.org/site/c.kkLUJbMQKpH/b.2521629/k.BF03/Home.htm

Program or outcome evaluation assesses the extent to which planned activities produce the desired outcome among a target population. Evaluation is considered and set up when the project is designed. Program or outcome evaluation assesses the extent to which planned activities produce the desired outcome among a target population. The PE model is led by the Planning and Evaluation Committee consisting of key representatives from the collaborating organizations Planning, Development and Communications is the staff support unit charged with assisting the organization in its efforts to improve its ability to become a self-correcting organization through planning, monitoring and evaluation efforts.
http://www.ccpfc.org/rd/eval_center.cfm

Importance: http://www.cdc.gov/eval/framework.htm

Examples:
www.phoenix.gov/ARTS/eval0405.pdf

http://www.afterschoolflorida.hhp.ufl.edu/evaluation_links.html

Models of Evaluation:
http://www.edtech.vt.edu/edtech/id/eval/eval_models.html

o Defining and addressing the need to change program evaluation methods that rely heavily on data gathering by postal mail to online instruments.
o What do you propose to do?
o What is my plan?
o What are my objectives: INPUT
o Involving stakeholders
• Model & Method: OUTPUT
o Description - How to do it?
o Evaluation Matrix
o Reasons for selecting particular foci and approaches
o What are the challenges and or roadblocks?
o What are the assessment instruments to be used?
o Measurement considerations & Data collection
• Evaluation: OUTCOME
o How well can we meet my objectives?
o Why I and my team are the best to perform the evaluation
o Emerging trends
• IMPACT:Conclusion, Summary and Recommendations
o Summarize what I learned from this experience.
o What would I recommend to others who would like to replicate my efforts?
o What would they need to be prepared for?
o What needs to be improved?
o Strengths and weaknesses
o Other issues