Course evaluation

A course evaluation is a paper or electronic questionnaire, which requires a written or selected response to a series of questions to evaluate the instruction of a given course. The term may also refer to the completed survey form or a summary of questionnaire responses.

They are a means of producing feedback that the teacher and school can use to assess the quality of instruction. The process of (a) gathering information about the impact of learning and of teaching practice on student learning, (b) analyzing and interpreting this information, and (c) responding to and acting on the results, is valuable for several reasons.^[1] They enable instructors to review how others interpret their teaching methods. The information can also be used by administrators, along with other input, to make summative decisions (e.g., promotion, tenure, salary increases) and formative recommendations (e.g., identifying areas where a faculty member needs to improve).^[2] Typically, these evaluations are combined with peer evaluations, supervisor evaluations, and results of students' test scores to create an overall picture of teaching performance. Course evaluations are implemented in one of two ways, either summative or formative.

Course evaluation instruments

Course evaluation instruments generally include variables such as communication skills, organizational skills, enthusiasm, flexibility, attitude toward the student, teacher–student interaction, encouragement of the student, knowledge of the subject, clarity of presentation, course difficulty, fairness of grading and exams, and overall student rating.^[3][4]

Summative evaluation

Summative evaluation occurs at the end of a semester, usually a week or two before the last day of class. The current students of the class perform the evaluation. Students can reflect on teachers' instruction without fear of punishment because course evaluations are completely confidential and anonymous. This can be done in one of two ways: either with a paper form or with online technology. Typically, in a paper-based format, the paper form is distributed to students while the teacher is out of the room. It is then sealed in an envelope, and the teacher will not see it until after final grades are submitted. The online version can be identical to the paper version or more detailed, using branching-question technology to elicit more information from the student. Both ways allow the student to provide feedback. This feedback is intended to help teachers assess the quality of their instruction. The information can also be used to evaluate a teacher's overall effectiveness, particularly for tenure and promotion decisions.^[5]

Formative evaluation

Formative evaluation typically occurs during the current semester, when changes can be made, although many institutions also solicit written comments on how to improve it. Typically, this form of evaluation is performed by peer consultation. Other experienced teachers will review one of their peers' instructions. The purpose of this evaluation is to provide the teacher with constructive criticism on their teaching. Generally, peer teachers will sit in on a few lessons and take notes on the teacher's methods. Later on, the team of peer teachers will meet with the teacher in question and provide useful, non-threatening feedback on their lessons. The peer team will offer suggestions on improvement, which the teacher can choose to implement.

Peer feedback is given to the instructor typically in the form of an open session meeting. The peers first reflect on the qualities of the instruction. Then they move on to areas that need improvement. Next, the instructor will offer suggestions for improvement and receive feedback on them.

Student feedback can be an important part of formative evaluation. Student evaluations are formative when their purpose is to help faculty members improve and enhance their teaching skills.^[5] The teachers may require their students to complete a written evaluation, participate in ongoing dialogue, or direct discussions during the course of the semester. The use of a 'Stop, Start Continue' format for student feedback is highly effective at generating constructive feedback for course improvement.^[6]

At the Faculty of Psychology of the University of Vienna, Twitter was used for formative course evaluation.^[7]

Criticism of course evaluations as measures of teaching effectiveness

Summative student evaluations of teaching (SETs) have been widely criticized, especially by teachers, for failing to measure teaching effectiveness accurately.^{[2][8][9][10]} Surveys have shown that a majority of teachers believe that a teacher's raising the level of standards and/or content would result in worse SETs for the teacher, and that students in filling out SETs are biased in favor of certain teachers' personalities, looks, disabilities, gender and ethnicity.^[11] The evidence that some of these critics cite indicates that factors other than effective teaching are more predictive of favorable ratings. To get favorable ratings, teachers are likely to present the content that the slowest student can understand, and consequently, the content is affected.^[12] Quantitative fields tend to receive lower student evaluations.^[13] Many of those who are critical of SETs have suggested that they should not be used in decisions regarding faculty hires, retentions, promotions, and tenure. Some have suggested that using them for such purposes leads to the dumbing down of educational standards. Others have said that the typical way SETs are now used at most universities is demeaning to instructors^[14] and has a corrupting effect on students' attitudes toward their teachers and higher education in general.^[15]

The economics of education literature and the economic education literature are especially critical. For example, Weinberg et al. (2009) find that SET scores in first-year economics courses at Ohio State University are positively related to the grades instructors assign but are unrelated to learning outcomes once grades are controlled for.^[16] Others have also found a positive relationship between grades and SET scores, but unlike Weinberg et al. (2009), they do not directly address the relationship between SET scores and learning outcomes.^[17][18] A paper by Krautmann and Sander (1999) finds that the grades students expect to receive in a course are positively related to SET scores.^[19] Isely and Singh (2005) find that it is the difference between the grades students expect to receive and their cumulative GPA that is the relevant variable for obtaining favorable course evaluations.^[20] Another paper by Carrell and West (2010) uses a data set from the U.S. Air Force Academy where students are randomly assigned to course sections (reducing selection problems).^[21] It found that calculus students got higher marks on common course examinations when they had instructors with high SET scores, but did worse when they took later courses requiring calculus.^[21] The authors discuss several possible explanations for this finding, including the possibility that instructors with higher SET scores may have concentrated their teaching on the common examinations in the course rather than providing students with a deeper understanding for later courses.^[21][22] Hamermesh and West (2005) find that students at the University of Texas at Austin gave attractive instructors higher SET scores than less attractive instructors.^[23] However, the authors conclude that it may not be possible to determine if attractiveness increases the effectiveness of an instructor, possibly resulting in better learning outcomes. It may be the case that students pay more attention to attractive instructors. Meanwhile, a 2017 lawsuit was filed on grounds of xenophobic discrimination in course evaluations at the University of Kansas, with Peter F. Lake, the director of Stetson University's Center for Excellence in Higher Education Law and Policy, suggesting this is no isolated incident.^[24]

The empirical economics literature contrasts sharply with the educational psychology literature, which generally argues that teaching evaluations are a legitimate means of assessing instructors and are unrelated to grade inflation. However, as in the economic literature, other researchers outside educational psychology have reported negative findings on course evaluations. For example, some papers have examined online course evaluations and found them to be heavily influenced by the instructor's attractiveness and willingness to give high grades in return for very little work.^[25][26]

Another criticism of these assessment instruments is that the data they produce are largely difficult to interpret for purposes of self- or course-improvement, given the number of variables that can affect evaluation scores.^[27] Finally, paper-based course evaluations can cost a university thousands of dollars over the years, while an electronic survey is offered at minimal cost to the university.

Another concern raised by instructors is that response rates to online course evaluations are lower (and therefore the results may be less valid) than those for paper-based in-class evaluations. The situation is more complex than response rates alone would indicate.^[28] Student-faculty engagement is offered as an explanation, where course level, instructor rank, and other variables lacked explanatory power.

See also

• Classroom management
• Educational assessment
• Educational evaluation
• Donald Kirkpatrick, founder of the 'Four Level Model' of training evaluation
• Ronald Ferguson (economist), a researcher who studied student evaluation of teachers
• Student behavior report cards

References

External links

• Weinstock, R. B. (2004). Quality control in the course evaluation process.