There are two books I’ve read sort of recently, From Counterculture to Cyberculture by Fred Turner and How the Hippies Saved Physics by David Kaiser, that supplement each other as a picture of how antimaterialistic culture of the SF Bay area in the late 1960s through mid-1970s produced a lot of nonsense, but also hugely important new impulses in hard technical fields. Silicon Valley grew out of an ethos of DIY back-to-the-earth-ism (hence the “Homebrew Computer Club”), while the Fundamental Fysiks Group at Lawrence Berkeley Lab, took the energy of enthusiasm for parapsychology and mysticism, and channeled it into revival of an inquisitive style of physics that rediscovered entanglement and Bell’s Theorem, and laid the groundwork for quantum cryptography and quantum information science.
Category: Technical
Compute the interest
Another comment based on Sharon Ann Murphy’s wonderful book on 19th century life insurance in the US: She describes an 1852 case in which the American Mutual Insurance Company tried to renege on a claim, where a preëxisting condition was found in an autopsy.
Not surprisingly, the jury sided with the beneficiaries; they “were out thirteen minutes, just long enough to compute the interest” on the original claim.
Indeed, the verdict is not surprising. What is most surprising, however, is that the jury computed the interest. I wonder how likely it is that a jury of twelve today would include even a single person capable of computing compound interest.
Christmas Demography
Whereever I have lived in my adult life, the city has been extraordinarily quiet from Christmas to New Year’s — indeed, the quiet starts somewhat before Christmas. The natural explanation is that people go away for the holidays. (Students obviously do, but it’s far quieter than even at other times when students are on vacation.) The problem is, they must go somewhere, so it can’t be that every place empties out. (Obviously, some of the apparent quiet is simply the absence of traffic from people going to work, shopping, etc. When shops and restaurants close down in late December because of lack of customers it’s a bit self-fulfilling.) So why is it that I’m always in the places that people flee for the holidays?
A common pattern is that younger people with children travel to their parents, in their old home towns. In general, if there is a pattern of migration from some places X to other places Y, the sort of people who move around (like myself) will tend to be living at Y. At Christmas, then, the migration is temporarily reversed, and people travel from Y to X. For a brief time, Y empties out and X gets full. (Even 2000 years ago Joseph and Mary had to go back to their home town for Christmas. And, as we know, the town was full up.) So, people like me notice that whereever they happen to be living is one of the places that empties out, because of the selection bias. People of my parents’ generation are generally living in places that take in visitors at Christmas, and so perhaps seem livelier than at other times of the year.
Sex education and the multiverse
I recently read and enjoyed David Deutsch’s book The Beginning of Infinity, a tour d’horizon of quantum physics and philosophy of science, brewed up with a remarkably persuasive idiosyncratic worldview, even if it does descend into a slightly cranky and increasingly ignorant rant on politics and economics by the end. This was my first introduction to the “multiverse”, which seems to be the modern version of the many-worlds interpretation of quantum mechanics. I was impressed at how cogent this picture has become since I last interested myself for quantum mechanics and its philosophical interpretations in my teens.
It might not be right, but it does lay down a marker against the Copenhagen interpretation — position and path don’t exist except when measured, wave-particle “duality”, etc. — which in comparison seems more like a counsel of despair than a physical theory in any meaningful sense.
In thinking about it, I realised that I’ve long had the feeling that the Copenhagen interpretation was more than anything the physics educator’s version of chastity education: not a real solution, but mainly a way to avoid dealing with parents yelling “Your teacher told you what?!”
Health selection bias: A choose your own preposition contest
Back when I was in ninth grade, we were given a worksheet where we were supposed to fill in the appropriate conjunction in sentences where it had been left out. One sentence was “The baseball game was tied 0 to 0, ——– the game was exciting.” Not having any interest in spectator sports, I guessed “but”, assuming that no score probably meant that nothing interesting had happened. This was marked wrong, because those who know the game know that no score means that lots of exciting things needed to happen to prevent scoring. Or something.
With that in mind, fill in the appropriate preposition in this sentence:
Death rates in children’s intensive care units are at an all-time low ————— increasing admissions, a report has shown.
If you chose despite you would agree with the BBC. But a good argument could be made that because of or following a period of. That is, if you think about it, it’s at least as plausible — I would say, more plausible — to expect increasing admissions to lead to lower death rates. The BBC is implicitly assuming that the ICU children are just as sick as ever, and more of them are being pushed into an overburdened system, so it seems like a miracle if the outcomes improve. Presumably someone has done something very right.
But in the absence of any reason to think that children are getting sicker, the change in numbers of admissions must mean a different selection criterion for admission to the ICU. The most likely change would be increasing willingness to admit less critically ill children to the ICU, which has the almost inevitable consequence of raising survival rates (even if the effect on the sickest children in the ICU is marginally negative).
When looking at anything other than whole-population death rates, you always have the problem of selection bias. This is a general complication that needs to be addressed when comparing medical statistics between different systems. For instance, an increase of end-of-life hospice care, it has been pointed out, has the effect of making hospital death rates look better. (Even for whole-population death rates you can have problems caused by migration, if people tend to move elsewhere when they are terminally ill. This has traditionally been a problem with Hispanic mortality rates in the US, for instance.)
What happens if you forget the key?
Courts in the US and the UK have recently been ruling that criminal suspects may be forced to reveal cryptographic keys that encode files that may include incriminating evidence. US courts have been divided on whether this infringes upon the otherwise absolute right to avoid self-incrimination. I’ve never taken that argument very seriously — it’s certainly not in the spirit of the right to refuse to assist in prosecuting oneself to allow people to hide documentary evidence of a crime, just because the revelation would be “speech”. But while people may be compelled to testify in court, and in some democracies may be required to assist police by correctly identifying themselves, it’s not usual for people to be compelled by law to reveal particular information, particularly when they may not know it. While perjury charges may be brought against those who testify falsely, the inevitable unreliability of memory makes perjury convictions difficult, and I thought impossible when the subject simply pleads ignorance rather than testifying to a falsehood.
In fact, the strongest argument for a right not to reveal a password is that it’s not the hidden data that are protected by the right against self-incrimination, but rather the admission that you know the password, hence are at least in some way in control of and responsible for them, that cannot be compelled. According to the Regulation of Investigatory Powers Act 2000 (that was apparently a banner year for civil liberties in the UK), “failing to disclose an encryption key” is an offence in itself. In 2009 a man was jailed for 13 months for refusing on principle to provide encryption keys to the authorities, despite the fact that he was not suspected of any crime other than not cooperating with the police.
I have encrypted volumes on my laptop hard drive — with old exam papers — whose passwords I’ve forgotten. I probably should delete them, but I haven’t gotten around to it, and maybe I’ll remember one of these days. Even if I did delete them, they’d still be there on my hard drive unless I took exceptional measures. So if customs officials ever took an interest in my laptop while I was entering the UK, I could end up in prison for up to two years. The only thing I could do to protect myself is either to destroy the hard drive, or have it erased, which is itself suspicious.
Unlike most other criminal offences, the offence of withholding a cryptographic key is impossible to prove, but also impossible to disprove. It is even impossible for anyone but the accused even to know whether or not there has been any offence. And if there has been no criminal offence — if the accused does not, in fact, know the key — there is no way to prove that. It is the democratic state’s version of the plight of the man being tortured for information that he does not have, so that he has nothing to offer to end the suffering.
Along these lines, I was wondering about the current state of the right to silence in British law, and there came a revelation in the form of the British authorities (oddly, the news reports are all vague about which authorities it was; presumably the UK Border Agency, but maybe agents from a secret GCHQ data-mining task force) detaining the partner of journalist Glenn Greenwald under schedule 7 of the Terrorism Act 2000. According to the Guardian,
Those stopped under schedule 7 have no automatic right to legal advice and it is a criminal offence to refuse to co-operate with questioning,
This is pretty frightening, particularly when these laws are being so blatantly abused to settle political grudges.
Longitudinal fables, ctd.: Is Julia shrinking?
I was commenting on how people like to turn age-structured information into longitudinal stories: If 80-year-olds buy more big-band recordings, and 20-year-olds more rap, we describe how people’s tastes shift as they age, from the hard rhythms of rap to the gentle lilt of swing. And I noted that the Obama campaign got itself into trouble last year trying to turn its age-specific policies into a longitudinal fable, called “The Life of Julia”. Looking at the pictures of Julia at different ages
I had the impression that Julia is shrinking as she moves into her forties.
More careful inspection of the pictures revealed that she is not shrinking (or not much); the main height change came when she stopped wearing high heels at age 37. But that got me wondering: should she have been shrinking? Or would that again have been confusing the cross section with the individual life course — the period with the cohort effect, in demographer jargon?
It’s certainly true that cohorts in America (and in many other prosperous countries) have been getting taller. US Civil war soldiers in 1863-4 averaged 5′ 7 1/2″. 50 years later the average height of young men had not changed significantly, but by 1955 the average height of young men was up to 5′ 9 1/2″ (and they were attaining their maximum height several years earlier). It’s not clear to what extent the trend has continued in the US — according to recent data, the average height of young male adults in the US is still about 5′ 9 1/2″ — though it clearly has in other countries that have seen a substantial improvement in children’s average nutritional welfare, such as Portugal, or the Netherlands, Italy, and Japan.
There is also a tendency for individuals to shrink as they age, from compression of the spine, particularly pronounced after age 60, and more extreme in women than in men. A sketch from this paper is included below. So, in fact, the hypothetical Julia should probably have been drawn about 2 inches shorter at age 67 than when she was 20. That’s about 3% — hard to tell from the silhouettes, with the changing hairstyle and all…
It’s funny, because I have seen height used as a paradigm example of where cross-sectional measures are misleading if you interpret them as cohort effects — narrating the changes within individual lives — but at least for the latter half of the 20th century in the US, the cross sectional data seem to give the right picture.
Stephen Wolfram’s longitudinal fables
There’s lots of interesting plots on Stephen Wolfram’s analysis of Facebook data, but what jumps out to me is the way he feels compelled to turn his cross-sectional data — information about people’s interests, structure of friendship networks, relationship status, etc. as a function of age — into a longitudinal story. For example, he describes this plot
by saying “The rate of getting married starts going up in the early 20s[…] and decreases again in the late 30s, with about 70% of people by then being married.” Now, this is more or less a true statement, but it’s not really what is being illustrated here. (And it’s not just the weird anomaly, which he comments on but doesn’t try to explain, of the 10% or so of Facebook 13 year olds who describe themselves as married.) What we see is a snapshot in time — a temporal cross section, in the jargon — rather than a description of how the same people (a cohort, as demographers would put it) moves through life. To see how misleading this cross-sectional picture can be if you try to see it as a longitudinal story of individuals moving through life, think first about the right-hand side of the graph. It is broadly true, according to census data, that about 80% of this age group are married or widowed. But it is also true that 95% were once married. In fact, if they had had Facebook when they were 25 years old, their Stephen Wolfram would have found that most of them (about 75%) were already married by that age. (In fact, about 5% of the women and 3% of the men were already in a second marriage by age 25.)
So, the expansion of the “married” segment of the population as we go from left to right reflects in part the typical development of a human life, but it reflects as well the fact that we are moving back in time, to when people were simply more likely to marry. And the absence of a “divorced” category masks the fact that while the ranks of the married expand with age, individuals move in and out of that category as they progress through their lives.
Of course, the same caveat applies to the stories that Wolfram tells about his (quite fascinating) analyses of structure of friend networks by age, and of the topics that people of different ages refer to in Facebook posts. While it is surely true that the surge in discussion of school and university centred at age 18 reflects life-phase-determined variation in interests, the extreme drop in interest in salience of social media as a topic is likely to reflect a generational difference, and the steep increase in prominence of politics with age may be generational as well. (I wonder, too, whether the remarkably unchanging salience of “books” might reflect a balance between a tendency to become less involved with books with age, cancelling out a generational shift away from interest in books.)
Screens or Weights?
I probably shouldn’t be spending so much of my time thinking about U.S. election polls: I have no special expertise, and everyone else in the country has lost interest by now. But I’ve just gotten some new information about a question that was puzzling me throughout the recent election campaign: What do pollsters mean when they refer to a likely voter screen? Continue reading “Screens or Weights?”
On the downgrade
Further reflections on non-transitive folk probability
Continuing my thoughts about zero-one probability from here, I come to the recent decision of Standard & Poor’s to lower their rating of US treasury debt. There are plenty of reasons to doubt their judgement, both because they’ve been absurdly wrong in the past (subprime mortgage backed securities were AAA, but treasury bills are risky?), because they can’t read budget estimates or can’t do basic arithmetic, because they are trying to project political trends, which they surely know even less about than about arithmetic, or because the people who work there are generally known to be pretty dim. But from a probabilist’s point of view what’s strange is the timing. Whatever you may think of the recent deal to avoid the US defaulting on its debt, it did avoid defaulting on its debt. Surely the likelihood of a default went down after the deal was passed. So why is the credit rating lower this week than it was last week? Now, this is all perfectly consistent with the view that S&P is not actually making a prediction of future default probability, but simply seeking the best opportunity to promote its wares. Certainly, the way they operate is not the like someone trying to give what will be perceived as neutral advice; they act more like central bankers, timing their announcements to try to move markets and (above all) seem relevant. They’re reminiscent of the folktale of the rooster who threatens to withhold his crowing, which inevitably will forestall the sun rise. The other animals plead with him to relent, but it’s a threat that only works as long as the rooster is modest enough to recognise that he can’t hold out forever. In the case of the US treasury bonds, S&P held out, and still the sun rose.
But there is something about their approach that seems to make sense to intelligent people, and not purely idiosyncratic. I’m reminded of Tversky’s famous conjunction fallacy, with studies seeming to show that people’s everyday probability intuitions don’t necessary satisfy the apparently inevitable law of conjunction: The probability of A or B must be bigger than the probability of A and the probability of B. Here we see intuitions of probability that don’t seem to satisfy the law of total expectation: If are possible future states of the world, and
is the probability of event A conditional on
happening, then the probability of event A now must be some kind of average of these conditional probabilities.
Common Infirmities 