Fifth Avenue has been New York’s most exclusive thoroughfare almost since the first section of the avenue, between Waverly Place and 13th Street, was laid out in 1824.
It’s easy to see why. Fifth Avenue was ideal in terms of privacy and comfort; it’s as far as possible from the industry of the Hudson and East Rivers and removed enough from the retail that crept up Broadway as the decades progressed.
Fifth Avenue also lacked a streetcar line or elevated train, so the slender avenue wasn’t clogged with traffic and crowds of strangers.
As the 19th century went on and the Gilded Age was in full swing, a Fifth Avenue address became even more sought after. Old money New Yorkers and new rich titans of industry built their mansions on what was dubbed ‘Millionaire’s Row’—from Fifth Avenue in the 50s to the stretch along Central Park in the 70s, 80s, and 90s.
The rows of brownstone mansions and marble chateaus arrived by the early 1900s, and images of these massive houses have come to symbolize the wealth made during the Gilded Age.

But what became of the shacks and shanties that formerly lined Fifth Avenue, especially the upper end, before it became a millionaire colony?
Fifth Avenue above 59th Street “at one time…was invaded by more than five thousand ‘wastrels,’ and was known as ‘Shantytown,’ and its queer inhabitants as ‘squatters,'” stated Fifth Avenue Old and New, a book published in 1924 to commemorate the 100th anniversary of the avenue.
Not all of the shanties were residential. A New York Times article from 1901 that focused on the eventual mansion Andrew Carnegie was building on Upper Fifth Avenue referenced the “relics” of another era, when this section of Fifth was an undeveloped road.
“This upper section of the avenue shows many strange contrasts, for alongside the palaces of millionaires are to be found old-fashioned roadhouses and buildings that are little more than shanties, relics of the former days of the avenue when it was a road,” the Times wrote.
A squalid shack next door to Carnegie’s stunning mansion at Fifth and 91st Street was the subject of another Times article in 1905.
“Within a stone’s throw of Andrew Carnegie’s mansion, the marble-colonnaded twin residence of G. L. and C. W. McAlpin, and the somewhat less pretentious home of Carl Schurz stands a gabled shanty within 20 feet of Fifth Avenue of such scant dimensions and poverty-stricken appearance that it would be despised among the hovels that house some of the poorest of the city’s residents.”
The Times goes on to describe the family of 5 kids headed by an Irish father who works as a stevedore (and their dog, an “ugly-tempered canine brute”). “The space of the dwelling that serves as home for those six human beings and the beast is probably something like 20 by 12 feet, divided into two rooms.”

What happened to this family, and other owners and inhabitants elbowed out of upper Fifth Avenue? There’s no follow up, but it’s safe to say their rickety homes, built on land they didn’t own, were condemned once the lot was sold and construction was to begin on another mansion.
“The owners of the land are simply awaiting purchasers at fancy figures, and meanwhile do not care what sort of building remains on the land,” the 1901 Times piece states.
Fifth Avenue wasn’t the only millionaire mile in New York City with shantytowns. Riverside Drive, lined with Queen Anne and Beaux Arts mansions in the early 1900s, was also dotted by shanties and squatter shacks.
[Top image: Corbis; second image: MCNY, MNY227520; third image: Wikipedia; fourth image: MCNY, MNY219239; fifth image: New York Times; sixth image: Fifth Avenue Old and New via Columbia University Digital Collections]