Tuesday, June 8, 2010

Optimal PhD core exam implementation

Suppose you are administering a core exam in a PhD program. Your material is such that you want a certain amount of material available to the students while they take this exam. Suppose you decided two years ago that you would let every student take into the exam 3 sheets of paper with students' notes on both sides. Now assume that a student's score is a function of how good and plentiful the information on his sheet is. You find that too many students are passing your minimum threshold because they are bringing in more notes than you thought they could on 3 pages. You realize they are doing this not by handwriting any smaller and smaller, but they are coming up with other ways to get more information on the page, typing it, coming up with shortcuts or abbreviations, etc. You do not want to change your threshold of passing score, but you want fewer students to pass. So you must reduce the distribution of scores by decreasing the amount of notes students are allowed to bring in.

Now you have three options I can see. The first way is to give everyone the same set of notes that the professors would prepare, but then everyone would get the same score, so they can't do that. The second is telling students they can no longer type, and they must write it by hand. The third way is by telling students they may now only bring in 1 sheet of paper front and back. This reduces the information by 2/3.

Which of option 2 or 3 should they choose. I propose they choose the third option. Not only is it easier to enforce, it also selects for more innovative and technologically savvy people compared to option 2. Option 2 on the other hand, is hard to enforce, and instead selects for people that can write small. I would argue in today's society, being technologically savvy and innovative has a stronger correlation with success in almost every field than being able to write small.

Indeed they have chosen option 3.