Dr. Fox effect


The Dr. Fox effect is a correlation observed between teacher expressiveness, content coverage, student evaluation and student achievement. This effect also allows insight to other related effects, such as those discussed below, and relationships between student achievement and evaluations of the teacher.

Experiment

The original experiment was performed at the University of Southern California School of Medicine in 1970 in which two speakers gave lectures to a classroom of MDs and PhDs on an irrelevant topic. The topic, "Mathematical Game Theory as Applied to Physician Education", was chosen to eliminate any chances the students being lectured would know something about the actual subject. Students were divided into two separate classrooms; one classroom would be lectured by an actual scientist and the other by an actor, Michael Fox, who was given the identity "Dr. Myron L. Fox", a graduate of Albert Einstein College of Medicine.
In the first half of the study the actor was instructed to teach his material in a more monotoned and inexpressive voice. This lecture was then compared to the control lecture by the scientist. After the lectures, the students were tested on the information they had learned. The students who attended the lecture taught by the scientist learned more about the material and performed better on the examination.
However, when both the actor and the scientist presented their material in an engaging, expressive, and enthusiastic manner, the students rated Dr. Fox just as positively as the genuine professor. This lack of correlation between content-coverage and ratings resulting from conditions of strong expressiveness became known as "the Dr. Fox effect".
In a critique of student evaluations of teaching, professor of law Deborah Merritt summarized the Dr. Fox effect as it was observed in the first experiments: "The experimenters created a meaningless lecture and coached the actor to deliver it 'with an excessive use of double talk, neologisms, non sequiturs, and contradictory statements.' At the same time, the researchers encouraged the actor to adopt a lively demeanor, convey warmth toward his audience, and intersperse his nonsensical comments with humor. The actor fooled not just one, but three separate audiences of professional and graduate students. Despite the emptiness of his lecture, fifty-five psychiatrists, psychologists, educators, graduate students, and other professionals produced evaluations of Dr. Fox that were overwhelmingly positive. The disturbing feature of the Dr. Fox study, as the experimenters noted, is that Fox's nonverbal behaviors so completely masked a meaningless, jargon-filled, and confused presentation."
A 1980 study found that prestige of research could even be increased by confounding writing style, with research competency being correlated positively to reading difficulty. Anecdotal evidence has since been reported by researchers.

Subsequent research

Study of Air Force Academy

In this study, students were assigned randomly to professors, to eliminate the possibility of good students receiving better professors. These professors were then given all the same syllabi, curriculum levels, and final testings so the difficulty was even for all groups. There was also a follow-up course given to the students to test the value of fundamental learning that the students received. These professors taught introductory calculus to a group of more than 10,000 students to achieve the proper information.
When the evaluations were processed, the professors who were less experienced and less qualified had the best evaluations and best performances on the final examination. However the students who attended the examinations given by the more qualified and experienced professors did best on the follow-up examination. This created speculation that the professors who were more experienced taught the material in a more general manner to produce a more fundamental understanding. This became obvious as a result of the follow-up examinations.
The results showed that the professors who instilled a more fundamental meaning of the material appeared worse on the initial examination and evaluations, but eventually obtained better academic results. This finding renders questionable the validity of student evaluations.

Eyal Peer and Elisha Babad

Criticisms of the original study include the experiment's lack of a control group, the use of a "Yes/No" scale, and lack of measurement evaluating learning among the participants. In a recent study published in the Journal of Educational Psychology, researchers Eyal Peer of Carnegie Mellon University and Elisha Babad of Hebrew University of Jerusalem recreated the original 1973 study taking these criticisms into consideration.
Through their research, they found that the Dr. Fox effect was still present despite the manipulations they added. Manipulations included the use of a more in-depth questionnaire, removal of the first thirty seconds of video describing the credentials of the lecturer, and a warning at the beginning of the questionnaire to ensure that participants responded honestly.
However, the researchers found that although the students with a charismatic teacher enjoyed the lecture more, they reported that they had not actually learned anything new. This is in contrast to the original study, which argued that people attending the lecture actually believed they were learning new material. From their research they were able to reason that an enthusiastic speaker may entertain an audience, but much more is required to be a successful teacher.

Halo effect

The Halo effect shares similar qualities to those of the Dr. Fox effect in relation to student evaluations of teachers. The halo effect is a cognitive bias in which our general impression of a person influences how we feel and think about his or her character. For example, attractive-looking people create a halo effect in which we perceive them as kind, smart or successful, but it may not be true because their attractive appearance interferes with our judgment of their performance capabilities. In one study examining halo effect on student evaluation, there was a better rating for teachers who provided more nonverbal immediacy, but the study also found out that a better evaluation of teachers was related to a greater halo effect. In the Dr. Fox effect study, a similar effect was found when Dr. Fox presented the lecture in an expressive manner.

Effects of instructor characteristics on student evaluations

From the experiment of Dr. Fox effect, the expressiveness of a teacher when delivering lecture material can affect student evaluation of the teacher. Other than that, individual differences among teachers such as personality, popularity, lecture fluency, non verbal behavior, and attractiveness can also affect student evaluation of teachers.

Personality

Instructor personality is one factor that has been shown to affect course evaluations. For example, in one study examining the big five personality dimensions, teachers who were perceived to be more extroverted, open, agreeable, and conscientious were evaluated more favorably, whereas teachers who were perceived to be more neurotic were evaluated less favorably. Further, Murray and colleagues suggested that the effects of instructor personality on student evaluations of instructor vary across the type of course. Although some traits, like a teacher's leadership, consistently predicted course evaluations, other traits varied across type of course. For example, a teacher's sociability positively predicted course evaluations in introductory psychology courses, but not graduate psychology courses, whereas the ambition of an instructor showed the opposite relationship – predicting evaluations of graduate, but not introductory psychology courses.

Charisma

The charisma or popularity of a teacher might also be a factor that contributes to teacher effectiveness and teaching quality. In one study that examined the assessment of charisma as a factor in effective teaching, charismatic teachers tended to receive good student evaluation and were also perceived as being more funny, helpful, encouraging, knowledgeable, sympathetic and other traits that are considered charismatic. Besides that, the study of Yun-Chen Huang and Shu-Hui Lin shows that different teaching methods can also increase the charisma rating of a teacher. For example, teachers that are perceived to be more charismatic offer explanations, answer questions by students, vary their teaching methods, and are also interested in and express concern for their students and their learning progress. Charisma of teachers is becoming more recognized recently with the introduction of an instrument to measure charisma, called "Inventory of Teaching Charisma in the College Classroom".

Lecture fluency

The fluency of a lecturer when delivering teaching materials can contribute to the effectiveness of student evaluation of a teacher. According to the study of Shana K. Carpenter and colleague that examine the effect of lecture fluency on learning perception, a lecture that is considered to be fluent suggested that the teacher stood upright and straight, maintained eye contact, and spoke fluidly without notes whereas reasons for considering a lecture disfluent include teachers that are slumped and stand with a bent back, look away, and speak haltingly with notes. Furthermore, the study also shows that fluent teachers that are prepared and well organized will receive better ratings than a disfluent teacher that is unprepared and disorganized in student evaluations of the teacher. Although the same study of lecture fluency on learning perception successfully shows that lecture fluency will increase the student rating of a teacher, further interpretation of the results suggests that lecture fluency biases the students’ perceptions of their own learning because lecture fluency did not actually affect the amount of information learned.

Non-verbal behavior

Non verbal behavior or non verbal communication is a series of wordless behavior that is projected by the speaker to the listener. Common examples of non verbal behavior include eye contact, smile, facial expression, distance between speaker and listener and any other wordless behavior that can communicate information between people. According to Virginia P. Richmond on 'Teacher Nonverbal Immediacy', teachers that show more positive nonverbal behavior increase the immediacy between students and also contribute to higher student rating. For instance, teachers with better rating by students were more likely to express non verbal behavior like smiling, walking around, head nodding and touching upper torsos whereas low rating teachers were more likely to touch their head, head shaking rather than nodding, and sitting on a chair. One study by Ambady and Rosenthal shows that subjects were able to form accurate judgement of impression by just watching short video clips of teachers providing non verbal behavior. An example from the study, participants were able to form accurate impressions of the teachers by just watching 10-second, 5-second, and 2-second lengths of short video clips about teachers walking to classrooms. Ambady research provides support that nonverbal behavior has a strong influence on impression formation of teachers.

Attire and appearance

The way that teachers present themselves or the attire they wear when teaching lectures can be a factor on student evaluation of teachers. According to Virginia P. Richmond, the attire of a teacher influences the way that students perceive their teacher. For example, teachers are perceived to be more competent, organized, prepared and knowledgeable when they dress formally with a coat and tie but they are also presumed to be not receptive to students' needs and low in student–teacher interaction. However, when teachers dress casually with collar-button shirts and jeans, they are perceived to be more friendly, flexible, fair and open but not so competent. From another study about predicting teacher evaluation from physical appearance, attire and appearance may be a factor influencing the rating by students but the effect will somewhat diminish when other more important and significant information is provided. For instance, the study found out that unattractive voices dilute the effects of attractive faces of teachers.

Background of Dr. Fox experiment

Authors

The actor portraying Dr. Fox was given one day to prepare for the lecture and was coached by the authors to present the lecture with "an excessive use of double-talk, neologisms, non sequiturs, and contradicting statements."

Motivation

There are previous reports and studies that show that the personality of a teacher/lecturer is one of the most important variables when it comes to evaluating the teacher's effectiveness. Getzels and Jackson, with support from Wallen and Travers, expressed that the personality and patterns of teacher behavior and methods presented represent the main forces for the most significant variable in evaluating the teacher's effectiveness. Goffman explains that the receptivity of the audience is strongly influenced by the person introducing the lecture, the quality of the introduction, and the speaker's involuntary expressive behavior, which becomes a decisive factor in how the audience responds to the conveying information. One study which looked at the student perceptions of educators from 7th to 12th grade and reported that the students regarded “teacher charisma or popularity” as the most important characteristic when rating teachers.
If the charisma or popularity of the teachers have such an effect on their effectiveness ratings by junior high and high school students, would the ratings be influenced in the same way with a group of well-trained professional educators in a learning situation? So what the authors were looking to see is that if a sufficiently impressive lecture was given, would it result in an experienced group of educators participating in a new learning situation feeling satisfied that they had learned from the lecture despite irrelevant, conflicting, and meaningless content conveyed by the lecturer.