Joanna's reflections from American Evaluation Association 2010

I was the primary presenter for one demonstration session at American Evaluation Association and I also assisted in preparing two additional presentations. The focus of my primary presentation was on cognitive lab methodology whereby you ask an individual to "think-aloud" as they engage in a decision-making process such as completing a survey item or solving a multiple-choice test item. This method provides information about the cognitive and affective processes that an individual engages in while making a decision and is often used to validate or improve research instruments. The audience that attended my demonstration included eight that were interested in conducting cognitive labs. Thus they posed several logistical and methodological questions. The audience members also provided helpful suggestions for improving cognitive labs in future research. The presentation was well received.

The most valuable presentation that I attended was think-tank session #605 led by Dr. Nathan Balasubramanian. This session was focused on defining and measuring teacher quality and effectiveness which is one of my primary research interests. Balasubramanian shared his work which has been aimed at creating a system for providing teachers with timely assessment data in his state. In many states, assessment data is obtained too late for teachers to use it to guide instructional decision-making. He has also found a way to report the data so that teachers can better evaluate growth over time by standardizing scale scores across grade levels. Balasubramanian invited colleagues from representatives from several schools in his state to discuss how they were using this information and how it has impacted instruction and achievement in their schools. The audience also encouraged Balasubramanian to consider other measures of teaching quality beyond standardized tests and there was considerable debate about the most important indicators of quality teaching and the most appropriate goals for K-12 instruction. The powerpoint slides for this presentation can be seen at http://comm.eval.org/EVAL/EVAL/Resources/ViewDocument/Default.aspx?DocumentKey=11a532a4-d321-417b-bffa-ac625fb1df56