Survival Bias

Hunter Mansion wasn’t the only grand old house demolished when Orchard Beach was created. The Marshall House, above, was built around 1820 on Rodman’s Neck, the piece of mainland south of Hunter’s Island that forms the bulk of the beach as it exists today. The house, also known as Hawkswood, was used for a while as a public building before being torn down in 1937.

The Bronx, as such, has had a bad reputation since the 1977 World Series and afterwards, when Jimmy Carter used it as a symbol of urban decay. As a result, most people today don’t associate the borough with lost grand mansions, although the Hunter and Marshall houses were built in the town of Pelham in Westchester County, when the name “Bronx” meant only the Bronx River.

There’s a fascinating issue in logic called “survival bias” where we judge data that has already been filtered. The classic example, which uses the word “survival” quite literally, is the illustration below.

During World War II, statisticians were examining the damage to bombers on their return from missions to see how they might be better protected. The drawing above is a composite showing damage that various planes survived. Abraham Wald made the argument that no additional armor was needed in the areas peppered with hits (red circles) because the planes had survived that damage. The areas without damage, on the other hand, needed to be armored because the evidence was that planes sustaining damage in those areas did not return from missions. That’s survival bias.

We don’t have a great picture of the building practices of the past because of survival bias, and the further back in time you go, the worse it gets. It’s common knowledge that the Old Law tenements in New York were built to make money and weren’t really intended to last all that long, even though they are all now 119 years old or older. When we – as professionals – look at these buildings today, we see low-quality masonry and carpentry work, but we’re looking at the survivors. There used to be a lot more of these buildings. Some were torn down because the land was more valuable with another building sitting on it, some were torn down because they were falling apart and were too expensive to repair. That last group is the interesting one: we do not today see the lowest-quality tenements because they’re gone. So our opinion of the quality of tenement construction is probably too good.

Or look at the surviving buildings from 1830 and earlier in New York. There’s a handful of old industrial buildings along South Street, a handful of churches, and a few houses. Most of the houses are farmhouses or country houses of wealth families, because they were located in the nearby villages that later became the outer boroughs. Anyone attempting to understand how New Yorkers lived in 1830 based solely on the surviving buildings would have a very warped view of what the city was then.

Or look at early skyscrapers. Most of those still present in Manhattan are located north of City Hall Park, even though the greatest concentration originally was south of the park. The ones to the south, in the Financial District, were built on land that was ultimately too valuable to be occupied by 10- or 15-story buildings, so the first generation of skyscrapers there was largely demolished for bigger and taller buildings. The pattern of the survivors is misleading.