|
| |
General Evaluation
Models/Frameworks |
|
General Evaluation Models/Frameworks
 | Management oriented |
| Consumer oriented |
| Expertise oriented |
| Adversary oriented |
| Naturalistic and participant oriented |
| Accreditation oriented |
| Research oriented |
| Decision oriented |
| Responsive oriented |
| Socio-political oriented |
| Social-psychological oriented |
Objectives Oriented
The
focus is on specifying goals and objectives and determining the extent to which they have
been attained
| Distinguishing Characteristics
 | measurable objectives |
| objective instruments |
| discrepancies between objectives and performance |
|
| Benefits
| focus on outcomes |
| ease of use, simplicity |
| forces objectives to be set |
| widespread acceptability |
|
| Limitations
| oversimplification |
| outcomes-only orientation |
| reductionistic |
| linear |
| over-emphasis on student testing |
|
| Criteria for judging evaluations
| measurability of objectives |
| measurement reliability and validity |
|
Management Oriented
The
central concern is on identifying and meeting the informational needs of managerial
decision-makers
| Distinguishing Characteristics
| serves rational decision making at all stages of program
development |
| evaluates at all stages of program development |
|
| Benefits
| comprehensive |
| sensitive to information needs of decision makers |
| systematic approach covering all stages of program |
| detailed guidelines for implementation |
|
| Limitations
| emphasis on organizational efficiency and production |
| assumption of orderliness in decision making |
| expensive |
| narrow focus on concerns of management |
|
| Criteria for judging evaluations
| utility, feasibility, propriety, and technical soundness |
|
Consumer Oriented
The central issue is developing
evaluative information on educational "products," broadly defined, for use by
educational consumers in choosing among competing curricula, instructional products, etc.
| Distinguishing Characteristics
| product information to aid decisions about educational
products and adoptions |
| criterion checklists to analyze products |
|
| Benefits
| emphasis on consumer information needs |
| influence on product developers |
| concern with cost effectiveness and benefits |
| availability of checklists |
|
| Limitations
| cost and lack of sponsorship |
| may suppress creativity or innovation |
| not open to debate or cross-examination |
|
| Criteria for judging evaluations
| freedom from bias |
| technical soundness and defensible criteria |
| evidence of need and effectiveness required |
|
Expertise Oriented
Depend
primarily on the direct application of professional expertise to judge the quality of
educational endeavors
| Distinguishing Characteristics
| based on individual knowledge and experience |
| team cite visitations and consensus standards |
|
| Benefits
| efficient, easy to implement, and broad coverage |
| use of human judgment |
|
| Limitations
| vulnerable to personal bias |
| lack of documentation and replicability |
| susceptible to conflict of interest |
| reliance on qualifications of experts |
|
| Criteria for judging evaluations
| use of recognized standards |
| qualifications of "experts" |
|
Adversary Oriented
Planned
opposition in points of view of different evaluators (pro and con) is the central focus of
the evaluation
| Distinguishing Characteristics
| use of public hearings |
| use of opposing points of view |
| decision based on presented arguments |
|
| Benefits
| close examination of claims |
| illumination of issues |
|
| Limitations
| fallible judges or arbiters |
| potential high cost and time consumption |
| reliance on communication skills of presenters |
|
| Criteria for judging evaluations
| balance, fairness, publicness |
| opportunity for cross-examination |
|
Naturalistic and Participant oriented
Naturalistic
inquiry and involvement of participants (stakeholders) are central in determining values,
criteria, needs, and data for the evaluation
| Distinguishing Characteristics
| reflecting multiple realities |
| inductive reasoning and discovery |
| firsthand experience |
|
| Benefits
| focus on description and judgment |
| wide variety of information and induction |
| emphasis on understanding and context |
|
| Limitations
| potential for over attention to atypical case |
| cost/labor intensive |
| possible failure to reach closure |
|
| Criteria for judging evaluations
| credibility, "fit", confirmability, auditability |
|
Accreditation Oriented
Provides
professional judgments the regarding quality of a program
| Distinguishing Characteristics
| relies on expert judgments (opinions) |
| examines program’s features rather than activities
and outcomes |
| process involves published standards, self-study, team
visit, site report, report review, and accreditation decision |
|
| Benefits
| reasonably broad coverage |
| expert, experienced opinions can be helpful |
|
| Limitations
| difficult to replicate |
| vulnerable to personal bias |
| validity depends on qualifications of experts |
| can be time-consuming |
| often seen as having little utility |
|
Research Oriented
Purpose
is to develop theory and generalizations and enable cause and effect statements
| Distinguishing Characteristics
| operationalize expected outcomes |
| random selection of participants |
| comparison group |
| random assignment of participants to groups |
| theory driven |
|
| Benefits
| technically sound (leads to cause and effect statements) |
| results can be generalized to other setting |
|
| Limitations
| difficult to use in applied settings |
| more control over extraneous variables the less results
generalize |
| focus fails to account for human variation from setting to
setting |
| often a rigid, preordinate design |
|
Decision Oriented
Provides useful information
to aid in decision making.
Most associated with Daniel Stufflebeam
| Distinguishing Characteristics
| looks at all stages of program planning and evaluation
process |
|
| Characteristics
| Context - What are the needs within the program
situation? |
| Input - What are the resources available to meet the
needs? |
| Process - How well is the program being implemented? |
| Product - What are the outcomes of the program? |
|
| Benefits
| comprehensive |
| integrates well with planning |
| provides useful, focused information |
|
Responsive Oriented
Purpose is to understand and
portray the complexities of a program and to be responsive to information needs that
relate to the issues involved
| Distinguishing Characteristics
| attempts to identify multiple stakeholders in program |
| looks at issues of concern to variety of stakeholders |
| tends to be descriptive of program activities and
provides vicarious experiences for decision makers |
| seeks to capture multiple realities and perspectives |
|
| Benefits
| focus on description helps put program into a context |
| evolving, emerging design rather than preordinate |
| qualitative data has meaning for many clients |
|
| Limitations
| time-consuming |
| sometimes a lack of closure |
| does not necessarily meet information needs of management |
|
Socio-Political Oriented
A social-constructivist approach
which attempts to construct a view of reality based on the multiple perspectives of
stakeholders
| Distinguishing Characteristics
| believes that evaluation is a social-political process |
| emphasizes identifying political issues and players |
| process involves considerable negotiation activities |
| attempts to empower those not usually involved in
decision making regarding programs |
|
| Benefits
| recognizes that programs exist within a political context |
| acknowledges that decisions about programs are made in a
political arena |
| attempts to involve the disenfranchised in the process |
|
| Limitations
| requires considerable interpersonal and consulting skills |
| can be extremely time consuming |
| may involve more sensitive issues than program planners
want to cope with |
|
Socio-Psychological Oriented
Purpose
is to provide useful information to relevant decision makers and stakeholders by
understanding the social and psychological factors affecting evaluation decisions
| Distinguishing Characteristics
| recognizes that decisions are made within a personal,
psychological context |
| identifies social influencers affecting decision makers |
| identifies psychological factors affecting information
use |
|
| Benefits
| acknowledges that all decisions are not made rationally |
| is responsive to the individual needs of decision makers |
|
| Limitations
| does not necessarily consider organizational context |
| needs to be expanded and integrated with other approaches |
|
Epistemology
| Objectivism
| uses data-collection and analysis techniques that yield
reproducible and verifiable results |
| evaluation procedures externalized |
| derived from empiricism |
|
| Subjectivism
| validity depends on relevance of evaluator’s
background and qualification and keenness of his perceptions |
| evaluation procedures internalized |
|
Utilitarian Evaluation
| Assess overall impact of program |
| Tend to follow objectivist epistemology |
| Focus on total group gains by using some common index of
"good" to identify the "greatest good for the greatest number" |
Intuitionist-Pluralist Evaluation
| Value depends upon impact of program on each individual |
| Tend to follow subjectivist epistemology |
| Focus on distribution of gains by individuals and
subgroups |
| No common index of "good" but a plurality of
criteria and judges |
| Evaluator no longer an impartial "averager" but
a portrayer of different values and needs |
    
| |
|
