What Does Educational Research Really Tell us?

Photo Credit Nic's Events

Most educators in the past 10 years have confidently followed the lead of Robert Marzano in his famous meta-analysis of What Works in School and attempted to implement some or all of those high impact strategies that his organization has uncovered by looking at hundreds of studies and averaging the effect of each study to determine which strategies have the greatest impact.  Therefore, summarizing and notetaking is considered an effective strategy because the average improvement of performance on the studies under consideration was strong.

I started wondering about the advantages and disadvantages of this approach as I was reading Creating the Opportunity to Learn by A. Wade Boykin and Pedro Noguera.  These authors used a different approach as they recommended promising practices to close the achievement gap.  They often cited research from a singular study (though they also referred to meta-analysis as well).  For example when discussing Teacher Student Relationship Quality (TSQR) they cited a study by Hamre and Pianta where the researchers followed 1900 Kindergarten students through 8th grade.  The relationship between the teacher and student was quantified in Kindergarten and their academic performance and grades were compared every year.

My wondering is what can we learn from these two approaches to research. It seems to me that the individual study gives us some specific lessons, especially if we dig into the manner that those Kindergarten teachers built productive relationships with their students, for example.  Whereas in the meta-analysis we are left with the general idea that students should be taught note taking and summarizing.  While I feel that this is a gain in our understanding, I'm guessing that within those many studies that were used for the meta-analysis there were students who gained, some who stayed the same, and a few who even dropped.  Therefore, we need to know more by looking at some of the individual studies with the biggest gains to determine the specific factors that led to success compared to those who did notetaking and summarizing but saw no student gains.

Either way, educators need be wise consumers of research, looking closely at the claims of researchers and practitioners alike.  We need to be able to judge for ourselves these claims and judge our own work through reflection and objective analysis to determine if what we are doing is working for the students we serve.