Skip to content

The Overdetermination of Choice -> The Overestimation of Effects

August 13, 2010

I suppose I should clarify why I think heterogeneous treatment effects are dire enough in education research to cast doubt on a lot of estimations of the value of a high school degree.

In my last post I only hinted at the first type of selection bias, whereas arguably type II is more important. Both forms of selection bias are mechanisms that determine what kind of person appears in our data set with a high school degree.

Selection Bias Type I: You know that you never want to work/could never hold down the kind of job that requires a high school degree. You therefore decide not to obtain a high school degree. The marginal utility for you of a high school degree is 0.

Selection Bias Type II: You attend an underperforming public school. You learn few valuable skills while sitting in the classroom and you realize that a diploma from Public School 128 is not a valuable credential, and so you drop out at a young age.

Now, if you attend a St. Grottlesex school, not only do your demographic characteristics almost guarantee that you’ll graduate, but your degree is also worth a lot more than other degrees, incentivizing you even further to graduate.

So what does this all add up to?

Students with family characteristics most predisposed to finishing high school are incentivized by the opportunity to obtain the most valuable degrees.

Students with family characteristics least predisposed to finishing high school are incentivized by the opportunity to obtain the least valuable degrees.

In the population as a whole, then, valuable degrees are overrepresented and less valuable degrees are underrepresented. So when researchers try to estimate the value of a high school degree with an unbalanced data set, they will consistently overestimate the actual value of a degree for the least advantaged groups.

The unfortunate consequence of this is that a John McWhorter can casually glance at poverty statistics and then wildly overestimate the efficacy of a high school degree. Armed with his cross-tabulation, he then says, “Look how valuable a degree is! When you choose not to graduate, you have chosen a life of poverty.” It’s a pat observation, and one that makes the alleviation of poverty seem like you just need a few more guidance counselors to twist a student’s arm here, give a word of encouragement there.

But I’d argue that to some (possibly very large) extent, the failure of so many students to graduate is a reflection of what those students estimate the value of that degree to be. If their school’s performance is poor and if they see few opportunities for high school grads in their community, they may well wonder what the point of showing up to homeroom each morning is.

In a world like that, it’s better to think of low graduation rates as symptoms – not the causes – of deeper social ills.


From → Uncategorized

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: