Teaching Critical Thinking

One question I grapple with is 'how do we get students to ask questions about, or rather to question, peer reviewed research papers?' This is based on my experience that undergraduate students and even many introductory graduate students have difficulty grasping the concept that there may be issues or even important problems with peer reviewed research. Part of this is based on an inherent appeal to authority/self confidence issue, how could a lowly undergraduate find something problematic with a paper written by Ph.D., or equivalently trained, scientists.

Why We Care About Critical Thinking

However, the question I am grappling with is just a subset of the more important issue, how do we teach students how to ask the 'right' questions. The key here being 'right.' This is a fundamental aspect of critical thinking. Being able to identify the assumptions, biases, controls needed, discrepancies, etc. in an argument, and a peer reviewed research paper is nothing if not an argument. I find the most successful approach is to identify these, and other, points by asking questions. Again these question have to be the 'right' questions.

In my advanced undergraduate class, I can usually classify my students into 3 categories: the non-questioners, the trivial questioners, and the rare critical thinking questioner. By the end of my course, I want my students to find themselves generally in this latter category.

In the first category, the non-questioner, we find the shy students who are uncomfortable speaking up. This silence could be the result of inherent shyness, poor classroom experiences, or even cultural issues. In fact, this point of 'cultural issues' reminds me that it is important for me to remember that women and minorities are frequently ignored or blatantly omitted from discussions. In my experience, there are as many if not more women promoting the discussions in my course as men. Regardless, I try to address the issue of cultural differences early by calling on women and minorities during our 

The Shy Student

discussion sessions. Included in this category of students are those who are not confident with the material and thus do not want to speak up for fear of saying something stupid. I provide many resources and tools to help bring students up to speed if they are missing some background, so I tend to be less sympathetic with these students because they, by definition, must be aware of their deficiencies and choose not to address them. Of course, it takes awhile to separate these students from those who are shy, but it is disheartening to identify a student as being  intellectually lazy, lazy in general, or indifferent. To be clear, I have had students that lack some of the foundational material needed for my course that have worked hard to address these issues, and I help them as much as possible, having one on one meetings as much as they need to go over concepts, specific papers, etc. I love these students, because they have a drive that is infectious. Getting back to the shy students, how can I help get them engage in the course, such that they can move to category 3 and without having to change their personalities? For these students, all students actually, I have online components to the course. In addition to in class discussions, I have an online forum to initiate new discussions or continue discussions started in the classroom. This provides a place for those students who are inherently shy and students who are not comfortable thinking on their feet, which is what the classroom discussions entail. Students can use these forums to ask broad questions, initiating discussions beyond the minutiae of the papers they read. Students can also ask for help if there is something in the papers, a method, conclusion, etc. they do not understand, and students can help their colleagues by answering those requests for help. While I monitor the discussion boards, I refrain from commenting as much as possible, such that it quickly becomes a student-driven environment. While not perfect, there are mechanisms to promote moving students from category 1 to category 3.

Were you there? Only applies
to science not the New Testament.

The second category: the trivial questioner, is the place I work the most. Not that a specific student is a constant trivial questioner, but rather it is a constant place we come back to in class. This is not a problem because it does serve as a constant 'teachable moment.' The trivial questioner falls into the meme that there are no stupid questions. Of course there are stupid questions! In fact, the 'there are no stupid questions' comment is itself a stupid comment. I understand the 'there are no stupid questions' concept, but it is used with the tacit understanding that everyone is acting in good faith. This is seldom the case. For example we have Ken Hamm's 'Were you there?' question. This question is bullshit and not acting in good faith. The fact he encourages ten year olds to ask this question just serves to exemplify the moral vacuum in which Hamm resides. Hamm knows this question is a bullshit question, but it is a nice soundbite gotcha-sounding question to the masses. However, the ten year olds Hamm sends out to 'ask' questions do not know why this is bullshit, and that is his goal. However, we can use it as a teachable moment. The problem, in my opinion, is that Hamm knows many children would never ask the question, but will think the question and then answer it for themselves. In my course, there are many 'were you there?' type questions. Not necessarily from the Hamm perspective, but from the 10 year old perspective of 'this sounds good, I'll go with it' perspective.' These kind of questions are particularly present at the beginning of the course and I like to think I help move the students into the 3rd category. It's possible that I push these students into the 1st category, but I doubt that based on the quality of the discussions as the semester progresses. By way of example, every year when discussing a paper using a mouse model, the question will arise 'well I am concerned that the study only used female mice and I wonder what the data would be if male mice were used?'  This kind of question is relatively easy to come up with because we teach students 'black and white' thinking, everything is a binary decision. So when the student reads the methods and materials and sees '20 female C57/B6 mice were....' the student immediately thinks 'male' or vice versa in a 'tell me the word that pop into your head when I say...' kind of way.

So how do I encourage questions/comments of these 2nd category students without pushing them into the 1st category? What I have found works is to mimic Socrates, I ask questions. For example...

I am concerned that the study only used female mice and I wonder what the data would be if male mice were used?

Why do you think this might make a difference? I agree that there are important differences between females and males, I'm wondering how you think these differences apply to this study?

Well there could be differences due to hormones or something...

That's a great point, because that is clearly the case in certain instances like Paracoccidiodes infections. Is there anything in this system that makes you think there is would be a sex-based difference?

.....

Ok that's a good point at face value, but maybe needs further consideration, did anyone have additional issues with this study?

The point is to encourage/require the students to have a scientific justification for their concerns, questions, critiques. This, in my opinion, is the most difficult thing the students can learn and that I can teach. The point is that I need to teach the students how to question the studies, but also to question their own questions/concerns. But, I want to emphasize that it is ok to be wrong! We talk about well conducted studies that have generally solid conclusions and identify potential concerns. These concerns may not change the overall conclusions, but do raise concerns with sub-conclusions that may not be valid. In fact, the introductory paper we are discussing is the one I railed on previously. We will also be considering the press release. This is a change from the last couple of years when we discussed a well written, described, and assessed paper. I'm interested in how this approach works.

Own this book!!! 

Finally, we come to the third category: the critical thinker questioners. For these students, I can only refine their skills, improve their writing, and expose them to new and interesting areas. Almost uniformly, these students have research experience and likely significant experience. However, these students almost certainly have holes based on the areas they have been exposed to. These students know the potential issues in the area they are familiar with, but lack the similar approach/mindset in the areas they are not familiar with. This is one of the reasons it is essential for scientists to read well outside their fields. Breadth of knowledge promotes a better assessment for how your studies fit into the broader world of science. This increases the impact of your research not only from the study in question, but also in the questions you ask in the first place.

FYI. While I have categorized student comments/questions into 3 groups, no one student (nor the instructor) fits into a given group. Furthermore, I love my course because I learn so much from the students, even those that tend to cluster in a specific category. The lessons I learn may vary, but I learn important concepts, holes, insights from all three groups.  I thank the students from previous years for helping me develop these insights and to improve my courses.