Not the Lake Wobegon Hospital

From the front page of the West County Times:

Death rates at Bay Area hospitals vary widely, new report reveals

While some hospitals excelled at keeping patients alive, more than half of institutions around the Bay Area had worse-than-average death rates for at least one medical procedure or patient condition in 2010 and 2011, a new state report reveals.

Longitudinal fables, ctd.: Is Julia shrinking?

I was commenting on how people like to turn age-structured information into longitudinal stories: If 80-year-olds buy more big-band recordings, and 20-year-olds more rap, we describe how people’s tastes shift as they age, from the hard rhythms of rap to the gentle lilt of swing. And I noted that the Obama campaign got itself into trouble last year trying to turn its age-specific policies into a longitudinal fable, called “The Life of Julia”. Looking at the pictures of Julia at different agesobama-julia-infographic

I had the impression that Julia is shrinking as she moves into her forties.

More careful inspection of the pictures revealed that she is not shrinking (or not much); the main height change came when she stopped wearing high heels at age 37. But  that got me wondering: should she have been shrinking? Or would that again have been confusing the cross section with the individual life course — the period with the cohort effect, in demographer jargon?

It’s certainly true that cohorts in America (and in many other prosperous countries) have been getting taller. US Civil war soldiers in 1863-4 averaged 5′ 7 1/2″. 50 years later the average height of young men had not changed significantly, but by 1955 the average height of young men was up to 5′ 9 1/2″ (and they were attaining their maximum height several years earlier). It’s not clear to what extent the trend has continued in the US — according to recent data, the average height of young male adults in the US is still about 5′ 9 1/2″ — though it clearly has in other countries that have seen a substantial improvement in children’s average nutritional welfare, such as Portugal, or the Netherlands, Italy, and Japan.

There is also a tendency for individuals to shrink as they age, from compression of the spine, particularly pronounced after age 60, and more extreme in women than in men. A sketch from this paper is included below. So, in fact, the hypothetical Julia should probably have been drawn about 2 inches shorter at age 67 than when she was 20. That’s about 3% — hard to tell from the silhouettes, with the changing hairstyle and all…

It’s funny, because I have seen height used as a paradigm example of where cross-sectional measures are misleading if you interpret them as cohort effects — narrating the changes within individual lives — but at least for the latter half of the 20th century in the US, the cross sectional data seem to give the right picture.

Longitudinal change of height with age
Longitudinal change of height with age

The Life of Julia: Another longitudinal fable

Picking up from my earlier discussion of the way cross-sectional data  get turned into (sometimes misleading) longitudinal stories, it’s been about a year since the Obama campaign unveiled The Life of Julia, a slide show that contrasted Obama’s and Romney’s policies with regard to their effects on women at different ages. Stated that way it would be pretty standard and uncontroversial, but in fact it turned into a flashpoint for the early part of the campaign. Why? Precisely because it was not a list of cross-sectional promises — What President Obama will do for children; What President Obama will do for seniors; etc. would be standard campaign web site headings — but was turned into a longitudinal story. These were not 12 different women, of different ages, who would putatively be helped by the president’s policies, but a single woman “Julia” who seems to be spending her whole life looking for government programs to scrounge from. Of course, it only seems this way because of the way this infographic interacts with our expectations of a biographical narrative, where we expect to be seeing the high points of her life, and every one of them involves government services. Creepy! It’s no wonder some critics were reminded of cradle-to-grave socialism.

Of course, the real story is cross-sectional. If Julia is 3 years old now, Obama is not really promising to provide a small business loan to her in the year 2040. And by the time she reaches retirement, she’ll probably be living on a Mars colony or hiding out from roving mutant bandits in subterranean bunkers after the nuclear climate catastrophe.

obama-julia-infographic julia full

Stephen Wolfram’s longitudinal fables

There’s lots of interesting plots on Stephen Wolfram’s analysis of Facebook data, but what jumps out to me is the way he feels compelled to turn his cross-sectional data — information about people’s interests, structure of friendship networks, relationship status, etc. as a function of age — into a longitudinal story. For example, he describes this plotrelationship-status-vs-age2

by saying “The rate of getting married starts going up in the early 20s[…] and decreases again in the late 30s, with about 70% of people by then being married.” Now, this is more or less a true statement, but it’s not really what is being illustrated here. (And it’s not just the weird anomaly, which he comments on but doesn’t try to explain, of the 10% or so of Facebook 13 year olds who describe themselves as married.) What we see is a snapshot in time — a temporal cross section, in the jargon — rather than a description of how the same people (a cohort, as demographers would put it) moves through life. To see how misleading this cross-sectional picture can be if you try to see it as a longitudinal story of individuals moving through life, think first about the right-hand side of the graph. It is broadly true, according to census data, that about 80% of this age group are married or widowed. But it is also true that 95% were once married. In fact, if they had had Facebook when they were 25 years old, their Stephen Wolfram would have found that most of them (about 75%) were already married by that age. (In fact, about 5% of the women and 3% of the men were already in a second marriage by age 25.)

So, the expansion of the “married” segment of the population as we go from left to right reflects in part the typical development of a human life, but it reflects as well the fact that we are moving back in time, to when people were simply more likely to marry. And the absence of a “divorced” category masks the fact that while the ranks of the married expand with age, individuals move in and out of that category as they progress through their lives.

Of course, the same caveat applies to the stories that Wolfram tells about his (quite fascinating) analyses of structure of friend networks by age, and of the topics that people of different ages refer to in Facebook posts. While it is surely true that the surge in discussion of school and university centred at age 18 reflects life-phase-determined variation in interests, the extreme drop in interest in salience of social media as a topic is likely to reflect a generational difference, and the steep increase in prominence of politics with age may be generational as well. (I wonder, too, whether the remarkably unchanging salience of “books” might reflect a balance between a tendency to become less involved with books with age, cancelling out a generational shift away from interest in books.)

Without a net

One linguistic phenomenon that fascinates me more than it probably should is when a word or phrase can have opposite meanings in different contexts. Like the English word cleave (e.g. Genesis 2:24 “Therefore shall a man leave his father and his mother, and shall cleave unto his wife”, as opposed to a meat cleaver.)

I recently watched Ken Burns’s controversial 15-hour documentary Jazz. One segment, focusing on Miles Davis’s turn to fusion and electronica, was titled “Tennis Without a Net”. This quotes the critic Gerald Early, who appears in the segment, but of course refers back to Robert Frost’s bon mot about free verse (“tennis with the net down”). The implication is that music is a game, whose spectators are judging above all the players’ adroitness in accomplishing inherently simple things under complicated artificial constraints. Free jazz has also been tagged with this undead witticism.

So playing “without a net” means making things too easy, too safe, since no one can say if you’ve gotten it wrong. But “performing without a net” can also mean taking exceptional risks, as in the title of the Grateful Dead’s 1990 album “Without a Net”. There the metaphor is the circus acrobat’s net, and the implication is that the band’s free improvisation is particularly risky, since they are performing live without the support of a predetermined musical structure.

And this reminds me again of Natalia Cecire’s fascinating attack on statistics as an “inherently puerile discipline”, because its highest priority is “commitment to the rules of the game”. (I should make clear, as I argued before, I disagree with Cecire’s opinion of statistics, but I find the framework she lays on it both creative and useful.) Are statisticians making their research too safe by performing with the net of mathematical methodology, or are humanists like Cecire setting themselves a too-easy task by playing with the net of rigorous quantitative analysis down?

What’s the Matter with Economics?

One of the most politically important economics results of recent years has been the paper by Reinhart and Rogoff on the link between high sovereign debt and low GDP growth. This work is something I’d been following for a while, as R&R’s book was one that I’d admired greatly. Their work claimed to show a strong negative correlation between sovereign debt/GDP ratio and ensuing GDP growth, and was reported as saying that 90% debt/GDP ratio marks a cliff that an economy falls off, killing future growth. This was seized upon by proponents of austerity as proof that budget cuts can’t wait.

As reported here and here by Paul Krugman, and here and here by Matt Yglesias, it now turns out that the result isn’t just theoretically misguided, it’s bogus. Economists who struggled to reproduce the results finally isolated a whole raft of errors and dubious hidden assumptions that completely undermine the conclusion. Only the most blatantly ridiculous fault was an error in their Excel spreadsheet formula that caused them to exclude important sections of the data from their computation. You’d think that this couldn’t get any worse, but instead of apologising abjectly, R&R have tried to argue that none of this was really essential to their real point, whatever that was.

My main thoughts:

  1. Do economists really do their analysis with Excel? I find this kind of shocking, like if I found out that some surgeons like to make their incisions with flint knives, or if airline pilots were calculating their flightpaths with slide rules. Once you accept that premise, it’s not surprising that they made a blunder like this. I’m not a snob about technology. Spreadsheets are great for doing payrolls, and for getting a look at tables of numbers, and doing some quick calculations. But they’re so opaque, they’re not appropriate to academic work, and they’re so inflexible that it’s inconceivable to me that someone who analyses data on a more or less regular basis would choose to use them. Continue reading “What’s the Matter with Economics?”

Statisticians of the World, Unite!

You have nothing to lose but your Markov chains!

From “A Brief History of Britain 1066-1485”, by Nicholas Vincent:

Finally, as in all modern debates from which statistics and the spirit of Karl Marx are never far distant, it has been argued that, by 1200, Philip of France was far richer than the King of England and therefore ideally placed to seize the Plantagenet lands.

Statisticians or Stakhanovites? You decide…

Screens or Weights?

I probably shouldn’t be spending so much of my time thinking about U.S. election polls: I have no special expertise, and everyone else in the country has lost interest by now. But I’ve just gotten some new information about a question that was puzzling me throughout the recent election campaign: What do pollsters mean when they refer to a likely voter screen? Continue reading “Screens or Weights?”

The Silver standard

Who speaks for statistics?Silver coins

Ace forensic psephologist Nate Silver has attracted quite a bit of attention lately, with his 4+-year-old blog devoted to his statistical model that is intended to provide a synoptic view of the entire range of public data to produce a single probabilistic prediction of the outcome. Now, there are some clear criticisms that could be made of his approach, and of his results — in particular, the obvious failure of his successive predictions to be martingales, as they would have to be if they were appropriately using all current information — but he has been remarkably clear and open about his procedures and principles, and his reasoning on matters large and small seems generally sound, if not necessarily compelling. It’s funny that his conclusions should arouse any controversy at all, given that they are hardly different (as Silver himself is quick to acknowledge) from the conclusions one would draw from a simplistic combination of poll results. His main contribution is in giving careful answers to the obvious critiques that could be proposed: What’s a reasonable estimate for the difference between state poll results and the actual election result? How correlated are polling errors? What’s the best way to average polls of varying qualities done over multiple days? And so on. In the end, the answer doesn’t differ much from what anyone with number sense would come up with in a few hours, but you don’t know that for sure until you do it. And Silver’s reputation derives from the sense and good care that he takes in posing these questions and resolving them.

(The failure of the martingale property is actually evidence of his honesty in following the model that he set up back in the spring. He clearly would have been capable of recognising the trends that other people can see in the predictions, and introducing an ad hoc correction. He didn’t do that.) Continue reading “The Silver standard”

Will small hospitals kill you?

Disquisition on medical statistics in The Guardian

A recent front-page article in The Guardian claimed to show that small NHS hospitals are killing people. “Huge disparity in NHS death rates revealed” was one headline. “Patients less likely to die in bigger hospitals“. “Safety in numbers for hospital patients” is another headline. The article makes no secret of its political agenda: “The results strongly suggest that smaller units should close. This presents a major challenge to the health secretary, Andrew Lansley, who has stopped all hospital reorganisation.” Online, Polly Toynbee decries “Hospital populism”, saying “Local hospitals may be loved, but they can kill.” Wow. That’s pretty bad. Here’s the schematic of the story: Smart and selfless experts want to save lives. Dumb public clings to habit (in the form of community hospitals). Evil politicians pander to dumb public, clings to campaign promises. “The health secretary, Andrew Lansley, has now put the project on hold, in line with his election promise to halt hospital closures, to the dismay of experts who believe that lives will continue to be lost.”
Continue reading “Will small hospitals kill you?”