Friday 30 October 2009

21947403 Evaluation Steps

Wednesday 14 October 2009

Logic Models ECUR 809 Assignment # 4

(A)Logic Model 1 - Flow Chart: General
Worksheet Flowchart 1

Logic Model 2 - Flow Chart: More Specific

21010006-WorksheetFlowchart-ECUR-809

B. Description of the Logic Model- Assignment # 4:
- scope of logic model (how much they cover);
- the number of levels included;
- the description of levels included;
- the direction of information flow
- the amount of text;
- the visual layout.
Each of these variables is described in turn, below.
Scope of Logic Model: The flow chart is a logic model designed for evaluation purposes of the whole programs at the Community Center. It begins with vision, mission, values, motivations, expectations, etc and at the end of the day, the purpose of evaluation is to find out if the programs are making the difference. The community center offers complex, multi-component programs that may require the development of separate logic models for each program component or activity. Thus, I designed one general and a more specific one that can help me to promote the need of evaluating ‘clients satisfaction’ in the community programs (which include Spanish & Tennis Teaching). Assessment and evaluation are best addressed from the viewpoint of the students’ reactions to 1) teachers &teaching, 2) class-assignments and 3) assignments-materials.
Number of Levels: The first flow chart logic model includes several ‘levels’ (goals, population of interest, long and short term objectives and indicators). The second one, includes strategies, activities, process indicators.
Description of Levels: There is no standard set of terminology for our logic models. So, the first one includes general terms and the second one applies more specifics. The discussion will begin with stakeholders:Who should be involved or engaged? Teachers, administrators, supervisors, coordinators, volunteers and students;the focus of my work will be on the students or participants of the Community Center Programs. How might they be engaged? Students will be invited in staff meetings, email, survey-questionnaires.
Direction of Information Flow: Both flow charts-logic models flows from moving from left to right starting with objectives and focus of the Evaluation: What am I going to evaluate?
The Community Center Programming (logic model 1) and the clients’ satisfaction – students’ reactions and satisfaction (logic model 2).
What is the purpose of the evaluation?
The purposes of this evaluation are to evaluate the extent to which (a) the organization and programs help the members of the Community in their personal and professional growth; (b) the participants or students are meeting the goals of the programs, namely the enhancement of student satisfaction and achievement.
Amount of Text: It is well known that the amount of text included in a logic model can vary greatly between logic models. It can be sparse and in point form, or highly detailed. As a matter of preference and the function, my logic models include the information needed for our purposes of presenting the most important issues:
Who will use the evaluation? How will they use the information?
- Teachers, administrators, supervisors, coordinators, volunteers and students.
- To assess the effectiveness of the programs and make changes and improvements to help teachers and students to meet the goals.
-To improve students achievement and satisfaction.
What questions will the evaluation seek to answer?
General questions:
Do Community Programs (CP) help participants or clients in their personal and professional growth and satisfaction? Are Programs meeting the goals set out by the Center? What are the reactions of students regarding those programs? Are they satisfied with their achievement of goals and performance?
Specific questions:
Does the community offer varied programs? Do teachers have adequate resources to implement them? Do they see growth in their students as a result of their CP?
Do Programs encourage students to develop personally and professionally?
Are Programs being used in the way that they are intended?
How are Programs perceived by students? Are they satisfied with their performance in those programs? What are the benefits to students?
Visual Layout: As we know there are many ways to approach visuals and overall layout. This is a highly subjective issue, but an important one as good visual design can greatly enhance the understandability of a logic model. In these cases I tried to avoid confusion and focus on the following questions:
What information do I need to perform evaluation? or to answer the questions?
Indicators – How will I know it? Level of satisfaction of participants.
When is the evaluation needed? At the beginning, in the middle and at the end of the program (s).
What evaluation design will you use? Consumer-Oriented Evaluation Approach.
Assessment and evaluation are best addressed from the viewpoint of the students’ reactions to 1) teachers & teaching, 2) class-assignments and 3) assignments-materials.
Collect the information:
What sources of information will you use?
Existing information:
Web site – Programs – Written materials provided by the Community Center – Teachers materials – samples of students’ work and/or experiences (videos, photos, etc).
People:
Teachers, administrators and the focus will be on students satisfaction.
What data collection method(s) will you use?
E-mail survey questionnaire to students - and teachers - (a larger sample).
Questionnaire- Interview (a small sample of four students).

About Assignment # 5- ECUR 809: The focus for my assignment will be on students’ testimonials of their experience with CP (a sample of four students).My focus will be on Programs: "Spanish Intermediate and Tennis Instruction"

Source: "Logic Models" Online:
http://www.thcu.ca/infoandresources/publications/logicmodel.wkbk.v6.1.full.aug27.pdf

Tuesday 13 October 2009

ECUR 809 Assignment # 3

ECUR 809 Assignment # 3: Evaluation of Organization -Performing an Evaluation Assessment.
Determining the feasibility and direction of my evaluation:
I have selected a community center as organization to use as a model for the rest of the course its programs (Adult General Interest programs such as Intermediate Spanish): http://www.gnag.ca/index.php
http://www.gnag.ca/index.php?page=154 I live close by; so, I can access individuals for input in my work. I chose the City of Ottawa: http://www.city.ottawa.on.ca/ specifically a neighborhood as organization because during the Spring/Summer I taught Spanish (complemented with tennis lessons)(www.moretennis.blogspot.com) as part of the "Ultra Play" program: http://www.ottawatennis.com/detail.php?news_id=294
Kindly please see below overview of my chosen organization:
Organization: A Community Center in the City of Ottawa, ON Canada
Program: "Adult General Interest - Spanish: Intermediate/conversational"
Model of Evaluation Assessment:student-centered evaluation Assessment.
According to the Student Evaluation: A Teacher Handbook(Saskatchewan Education, 1991) student evaluation should focus on the collection and interpretation of data which would indicate student progress. This, in combination with teacher self-evaluation and program evaluation, provides a full evaluation. Chapter one states that, "Assessment and evaluation are best addressed from the viewpoint of selecting what appears most valid in allowing students to show what they have learned." In general, the main phases are the following: preparation, assessment, evaluation (formative, diagnostic, and summative) and reflection. Below each one is briefly described:
Preparation: what is to be evaluated, the type of evaluation (formative, summative, or diagnostic) to be used, the criteria against which student learning outcomes will be judged, and the most appropriate assessment strategies with which to gather information on student progress. Decisions made during this phase form the basis for planning during the remaining phases.In the Spanish Intermediate and Conversation Program the criteria and strategies are guided by an instructor (graduate student) from the University of Ottawa.
Assessment: identify information-gathering strategies, construct or select instruments, administer them to the student, and collect the information on student learning progress. The identification and elimination of bias (such as gender and culture bias) from the assessment strategies and instruments, and the determination of where, when, and how assessments will be conducted are important considerations. Performing an evaluation assessment process of the Program "Adults General Interest," Spanish Intermediate and Conversation Program in the Community Center, City of Ottawa, requires an appropriate approach. The Stake's "responsive" approach seems to be an an adequate way to reporting the "success and failure" of that program. Stake (1975, p.19)recommended the "clock" model to reflect the prominent recurring events in a responsive evaluation: talk with clients, program staff, audiences; identify program scope; overview program activities; discover purposes, concerns; conceptualize issues, problems; identify data needs re issues; select observers, judges, instruments, if any; observe designated antecedents, transactions and outcomes; thematize: prepare portrayals, case studies; validate, confirm, attempt to dis confirm; winnow, for audience use; and assemble formal reports, if any. In this sense, the Stake's model helps in reporting evaluation assessment of Intermediate & Conversation Spanish Program, in which not only questionnaires but also specific tests and sample work portfolios were assessed. See example of past questionnaires.
Evaluation: the information gathered during the assessment phase is used to make judgments about student progress. Based on the judgments (evaluations), decisions about student learning programs are made and reported to students, parents, and appropriate school personnel.
Reflection: allows pondering the successes and shortfalls of the previous phases. Specifically, evaluate the utility and appropriateness of the assessment strategies used, and make decisions concerning improvements or modifications to subsequent teaching and assessment. Instruments contain questions that encourage reflection on student assessment,teachers'planning, and on the structure of the curriculum. In the Intermediate & Conversational Spanish, successes of the program of Intermediate and Conversational Spanish, we can mention the following: excellent Audio CD cassettes; and an exciting vacation with a great learning opportunity, offered in combination with similar programs to study Spanish complemented with other programs, such as sports tennis and golf, games and with latin dance programs.
Until now no failures have been reported. To the contrary, students are looking for more "living spanish" programs.

Sources: Program Evaluation, Particularly Responsive Evaluation (Occasional Paper No. 5, p.19) by R.E. Stake, 1975b, Kalamazoo, MI: Western Michigan University Evaluation Center, Adapted by permission. Cited in pag. 138 in Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines. White Plains, NY: Longman.
Online: www.wmich.edu/evalctr/pubs/ops/ops05.pdf
"Student Evaluation: A Teacher Handbook" Retrieved September 24th, 2009 Online:
http://www.saskschools.ca/~ischool/Drafting10/curr/part25.htm
Flowcharts:
http://www.scribd.com/doc/21010132
http://www.scribd.com/doc/21010006