Return on Investment

The TF program for the last two years has funded about 10 TFs at $4k / year, so around $40,000 a year.  Of necessity, that’s $40,000 that wasn’t spent doing something else, so it’s reasonable to ask whether we are getting maximum bang for those bucks.  That’s what a few recent posts, like the “Here’s $40k, design a K-12 intervention” series are about.

In this post though, I want to ask a little different question -- what constitutes a good bang for the buck?  I know every TF takes the position for different reasons (and in another post I’ll talk about my motivations), but I want to focus on the other side of the equation: is this a good use of UM’s money?  Trying to answer this question, I made a rather startling realization that I’m not certain what UM’s goals are with this program.  So I’ll make a reasonable estimate of what I think they are:

1. Increase the number of Ypsi students enrolled at UM, esp. in CoE.
   • Currently this number is ~5 across all years.  For contrast, my high school in Plymouth sent around 50 students to UM my year, maybe 20-30 to CoE that year alone.
2. Be able to point to the TF outreach program in good faith when writing the ‘broader impacts’ section of NSF grant proposals
3. Make a connection to a K-12 district, and figure out what the best way is for universities to integrate into the K-12 community

The first two are the clearly numeric parts, where you can probably draw out some dollars and cents analysis.  The third may feel like vague bureaucratese, but there is a pretty big disconnect between the public K-12 school districts and the state universities.  I remember being shocked at how hard you had to work in undergrad to succeed, and I came from a pretty good high school.  I sure didn’t know what I was going to major in, and if not for older siblings who’d already been to college I might not have known which classes to take in high school to prepare myself.

So, are these good goals?  Let’s start with increasing YHS enrollment in CoE.  If UM spends $40k a year on TFs, and as an optimistic result let’s say they get an average of 4 YHS students a year into CoE, that’s $10k per student, not counting any scholarships in the mix. 

On NSF funding, current numbers pulled from NSF’s website list, among currently active awards, $373 million awarded to date to University of Michigan PIs (nicely done Ken Wise with $34M to date!).  If, in bulk, this $40,000 effort translates into even a 1% increase in that overall funding level due to increased acceptance rates for CoE proposals, the $10k number above pales in comparison, and we would say this is a very good bang for the buck.

Of course, NSF likes to see quantifiable metrics of success in the broader impact area, and they are sadly unlikely to consider their own previous funding as evidence of success to increase funding further.  Alas, circular funding logic fails – you need concrete results to sustain the momentum.  This is where I should note that I think goal #1 springs from goal #2 as sort of a proxy metric, lacking anything better, but it’s kind of cheating when you consider the scholarships I mentioned (and then promptly neglected) before.  See, the last few Ypsi High students at UM have basically been on full rides, which literally means we’re paying them to come here.  So does that mean we’ve made an impact and broadened the pool of college-bound Ypsi students, or just convinced proportionally more of the same size pool to come here?  No good answer here, I don’t have enough data. 

So, goal 3, the K-12 connection.  UM has indeed built a connection to YHS.  What passes over that bridge?  YPSD is currently in their last year of NCLB probation, and at the end of this year barring a serious course change will to enter the total reboot phase where they break the teachers’ union, fire just about everybody, rehire some, and start fresh.  My father is a teacher and president of his AFT union local, so understand that I am more than a bit perplexed by the heavy-handedness of that action.  And yet, having spent two years up close and personal at Ypsi High, I am also deeply skeptical about the status quo there producing meaningful results lacking this sort of massive boot-meets-rear kind of action.

Anyway, the point is that if these are the metrics of success, probably goal 1 is coming along pretty well, since  I know of at least 3 students who are entering UM this year from Ypsi High.  Goal 2 is going great if my totally out-of-nowhere numbers are correct to within a few orders of magnitude, but is not sustainable without much clearer observed performance improvements, since NSF likes return on their investment (hence the title of the post).  Goal 3 is an area where I think we’ve made good progress, not necessarily in finding the solution but in identifying the relative effectiveness of a couple of ideas like the TF program, which will be undergoing substantial refinement between now and next year.

In close, we’ve kind of skirted the issue of whether this is the “best bang for the buck.”  I think that’s fair, because a) I lack good numbers, and b) this is an open question without a good answer yet anywhere despite lots of really smart people working on it across the country, so give me a break already.  But in terms of using our money wisely and in line with the goals above, esp. #2, I think the course of action is

1. Draw out lessons learned from the TF program and present a plan to improve it next year
2. Clarify what the metrics of success are that are driving that refinement and how we’re going to measure them
3. Do some legwork and make sure the brainstormed metrics we come up with are good ones according to the research that’s out there right now

Now, apparently somebody somewhere agrees with me, because #1 and #2 are what tomorrow’s TF meeting is directed toward.  Of course, #3 is probably above my pay grade, falling squarely in the (OE)^2 office and the CoE leadership.  Stay tuned.