TechTweets


	Maggie Koerth @maggiekb1 · Nov 08 I've been thinking about this for a day and a half now. And here's why. This game exists because a metric (selectivity) exists. And metric exists because ... once upon a time a newsmagazine decided they could make some good side money ranking colleges. https://t.co/jlGQUfPFf3
	Maggie Koerth @maggiekb1 · Nov 08 Does "selectivity" actually tell you anything useful about how good your education will be? What does "selectivity" actually measure that is of value to a student? Why do I have the feeling somebody chose this metric cause they just needed more stuff to rank by?
	Maggie Koerth @maggiekb1 · Nov 08 A few years ago, I read a research paper that used the US News and World Report college rankings as an example of how we take subjective arbitrary metrics and adopt them as representative of objective worth and then reshape whole systems around them.
	Maggie Koerth @maggiekb1 · Nov 08 And it boils down to this: You could make a useful-to-students guide to colleges. But that takes a lot of reporting. Because many of the things students want to know are subjective and can't be well captured in numbers. And you have to do it, yourself, over and over.
	Maggie Koerth @maggiekb1 · Nov 08 Or, you could come up with a bunch of metrics that colleges submit to you and that look scientific and objective. And that's a lot easier. Requires fewer reporters. Makes the universities compete for your favor in ways you can maybe monetize. So why not?
	Maggie Koerth @maggiekb1 · Nov 08 That, my friends, was an editorial decision. An editorial decision that has had broad-ranging effects on how kids feel about themselves, how schools treat them, what schools feel pressured to spend $ on (and raise tuition for) ...
	Maggie Koerth @maggiekb1 · Nov 08 And I guess the big takeaway for me is to spend more time thinking about the downstream effects of editorial decisions. When I make a choice, what could the consequences of that choice be?
	Maggie Koerth @maggiekb1 · Nov 08 Well, that, and my usual rant about "just because it's data doesn't mean it's objective fact that tells you what you need to know".
	Maggie Koerth @maggiekb1 · Nov 08 Relatedly, parents, keep this shit in mind when you're thinking about Great Schools rankings in your area. Ask questions about the metrics your public school is judged by. And then think about whether they are really telling you what you want to know.
	Maggie Koerth @maggiekb1 · Nov 08 At one point, I was looking at a house in a neighborhood whose grade school at a "2" ranking with Great Schools. And I got all worried. And I looked into it. It had low test scores because more than 50% of its students were ESL. But the teachers were winning awards.
	Maggie Koerth @maggiekb1 · Nov 08 The principal was beloved. It had some really cool after school robotics and language and arts stuff. It was not a bad school. It was a school that scored poorly on a ranking system.
	Maggie Koerth @maggiekb1 · Nov 08 I once literally heard the principal at my kids' current school tell a prospective parent that part of why our school didn't score as high as another school across town was because we didn't hire a person whose sole job was to massage the Great Schools ranking.

Maggie Koerth @maggiekb1 · Nov 08

I've been thinking about this for a day and a half now. And here's why. This game exists because a metric (selectivity) exists. And metric exists because ... once upon a time a newsmagazine decided they could make some good side money ranking colleges. https://t.co/jlGQUfPFf3

Maggie Koerth @maggiekb1 · Nov 08

Does "selectivity" actually tell you anything useful about how good your education will be?

What does "selectivity" actually measure that is of value to a student?

Why do I have the feeling somebody chose this metric cause they just needed more stuff to rank by?

Maggie Koerth @maggiekb1 · Nov 08

A few years ago, I read a research paper that used the US News and World Report college rankings as an example of how we take subjective arbitrary metrics and adopt them as representative of objective worth and then reshape whole systems around them.

Maggie Koerth @maggiekb1 · Nov 08

And it boils down to this:
You could make a useful-to-students guide to colleges. But that takes a lot of reporting. Because many of the things students want to know are subjective and can't be well captured in numbers. And you have to do it, yourself, over and over.

Maggie Koerth @maggiekb1 · Nov 08

Or, you could come up with a bunch of metrics that colleges submit to you and that look scientific and objective. And that's a lot easier. Requires fewer reporters. Makes the universities compete for your favor in ways you can maybe monetize. So why not?

Maggie Koerth @maggiekb1 · Nov 08

That, my friends, was an editorial decision.

An editorial decision that has had broad-ranging effects on how kids feel about themselves, how schools treat them, what schools feel pressured to spend $ on (and raise tuition for) ...

Maggie Koerth @maggiekb1 · Nov 08

And I guess the big takeaway for me is to spend more time thinking about the downstream effects of editorial decisions. When I make a choice, what could the consequences of that choice be?

Maggie Koerth @maggiekb1 · Nov 08

Well, that, and my usual rant about "just because it's data doesn't mean it's objective fact that tells you what you need to know".

Maggie Koerth @maggiekb1 · Nov 08

Relatedly, parents, keep this shit in mind when you're thinking about Great Schools rankings in your area. Ask questions about the metrics your public school is judged by. And then think about whether they are really telling you what you want to know.

Maggie Koerth @maggiekb1 · Nov 08

At one point, I was looking at a house in a neighborhood whose grade school at a "2" ranking with Great Schools. And I got all worried. And I looked into it. It had low test scores because more than 50% of its students were ESL. But the teachers were winning awards.

Maggie Koerth @maggiekb1 · Nov 08

The principal was beloved. It had some really cool after school robotics and language and arts stuff.

It was not a bad school. It was a school that scored poorly on a ranking system.

Maggie Koerth @maggiekb1 · Nov 08

I once literally heard the principal at my kids' current school tell a prospective parent that part of why our school didn't score as high as another school across town was because we didn't hire a person whose sole job was to massage the Great Schools ranking.