Overview of Evaluation
When I use evaluation I am keenly aware that evaluation has many aspects to it. There are many ways to describe the process of evaluation. The student achievement testing that we do so often is only one aspect of a much larger pie. Any form of gathering information, analyzing that information and arriving at a conclusion based on that information is called evaluation. One can think of formal and informal evaluation. When you look out the window to see what kind of weather the day may become, and when you use that information to choose the kind of clothes to wear on that day, then you are conducting an evaluation. If there are snow flurries outside, then you will choose clothing that can handle colder weather to keep you from freezing. You are using the information to determine what the weather might turn out to be like, and therefore what type of clothing to wear under those harsh conditions. The same is true on a bright clear day in the summer. This is informal evaluation and almost everyone performs some kind of informal evaluation at some point in their lives.
Another large category of evaluation is formal evaluation. This is characterized by formal enquiries mostly written down on paper or coming over the internet or the telephone. A major distinguishing characteristic here is that information is written down, presented in written format. Information appraisals, analyses and results are similarly written down, communicated in written messages. Formal evaluation has many aspects to it as well.
One attempt at a definition is to look at formal evaluation as the systematic determination of merit, worth, and significance of something or someone using criteria against a set of previously set standards. On many occasions evaluation is used to characterize and appraise subjects of interest in a wide range of human enterprises, such as criminal justice, the arts, profit and non-profit organizations, government agencies and programs, and other human activities. Evaluation is also used in industry to appraise, maintain quality and produce goods that meet certain standards. There is no one definition of evaluation that is acceptable to all practitioners.
There are many types of evaluation. Some of these are as follows:
· Formative evaluation is a type of evaluation whose purpose is to improve a program or a project while it is being implemented.
· Summative evaluation is that type of evaluation that that is typically performed at the end of a program or project or cycle.
· Course evaluation is that type of evaluation that is used to appraise instruction of a given course.
· Educational evaluation is evaluation conducted specifically in educational settings.
· Program evaluation is used to determine the worth, merit or significance of a program. Programs are usually of long-term duration, such as K-12, elementary to middle school programs, etc.
· Project evaluation is used to determine the usefulness, worth or merit of a project, usually of shorter duration, such as a few weeks or months.
So, basically, when one gathers and analyzes information for the purpose of making a decision, then one is conducting an evaluation.
Evaluators use many different ways to compile or collect information for later analysis. Some of the ways in which evaluators collect and assess information include the following:
Action Research, Benchmarking, Clinical Trial, Cost-benefit Analysis, Electronic Portfolio, Field Experiment, Grading, Marketing Research, Multivariate Analysis, Participant Observation, Policy Analysis, Quality Audit, Questionnaire Construction, Rubrics, Structured Interview, and Student Testing.
As an educator, it is usually instructive to be aware of what type of evaluation activity we are engaged in when we test students. In class we went into detail about what we can do to improve the evaluative effort. We should be aware of what limitations we will encounter as a result of the type of evaluation we are conducting.
Professional Evaluation
There is an international organization for professional evaluators. It is the American Evaluation Association (http://www.eval.org/), one of the foremost organizations in the profession. Its mission is to “improve evaluation practices and methods, increase evaluation use, promote evaluation as a profession, and support the contribution of evaluation to the generation of theory and knowledge about effective human action.” Its vision is “to foster an inclusive, diverse, and international community of practice positioned as a respected source of information for and about the field of evaluation.” Its values include “excellence in evaluation practice, utilization of evaluation findings, and inclusion and diversity in the evaluation community.”
Evaluation is a thriving profession in the US and elsewhere. A brief look at the companies whose members are affiliated with the American Evaluation Association (AEA) is shown at the following website: http://www.eval.org/find_an_evaluator/evaluators_found.asp. There is an informal microblog (http://eval.org/aeaweb.asp), which shares resources and updates regarding evaluators, evaluation, and the AEA community at large. Professional evaluators are involved in a wide range of activities on many facets of our social life. The AEA is in touch with a number of social media such as Evaluation Talk, Facebook, AEA365Blog, RSS, Twitter, Linkedin. 
Anyone may become 
a “fan” on any of these media. This may be seen as an attempt on the part of AEA to reach out to the community. (You may be taken to the websites if you click on these apparently empty squares.)
Evaluation and Personal Views
When I use evaluation to provide estimates of how much students have learned, I am all too aware of the limitations of obtaining information about a person when so many variables may alter the observed information. Just like we discussed in class, the observed score is an estimate of the true score. It is only when one removes all error scores from the observed score will one obtain a true score. That seldom happens, if ever. The more accurate the instrument is, the better prepared the student, and the more accurate the estimate will be. In general, therefore, it is always prudent to treat the scores as approximations of a person’s academic or cognitive or affective or psychomotor behavior.
With reference to Career and Technical Education, I would say that perhaps evaluation should be used more to document evaluation information and to present findings. When we talk of shrinking budgets at Congress, it may be useful for career and Technical Education practitioners to hire one or more of the professional evaluation firms to conduct formative or summative evaluations and to present such information to Congress. Congress may take a good second look at the proposals to increase funding when objective evaluative data is presented to them. Such evaluations are termed external evaluations because they are conducted by a consortium which has absolutely no ties with the parties requesting the evaluation.
Another way to tackle the problem of presenting credible information to Congress is to train and hire our own evaluators who can work with the CTE establishment. It is usually less expensive to train and hire your own people who can use the same techniques used by external evaluators. These would be called “in-house” evaluators because they are hired by the same parties that request the evaluation. I think we have a better chance of convincing Congress about the need to increase funding for CTE projects when we regularly present performance data to them. There seems to be a good case for the CTE establishment to seriously consider the role of professional evaluators in the future administration of the profession. The needs of the hour call for a different approach to be employed. This may be a profession worth embracing if the future of Carl D. Perkins funding is to be assured.
References
1. Formative evaluation
http://en.wikipedia.org/wiki/Formative_evaluation
2. Definition of evaluation
http://en.wikipedia.org/wiki/Evaluation#Methods_and_techniques
3. Evaluation methods and techniques
http://en.wikipedia.org/wiki/Evaluation#Methods_and_techniques
4. American Evaluation Association (AEA)
5. AEA Web – Evaluation Esoterica
No comments:
Post a Comment