REAL ESTATE TOPICS...What more than 1 million Craigslist rental listings tell us about the housing market

By Emily Badger

Craigslist, with its drab gray interface and homemade classifieds, has become the single largest information exchange about the rental housing market in the United States. Its digital bulletin boards have everything: apartment porn for places you'll never afford, weird fish-eye photos by amateur landlords, queries for every conceivable living space from a spare bunk to a full-sized mansion.

The site touches both the high and low ends of the market — the mom-and-pop operation and the professionally run high-rise — across hundreds of locations. And so Craigslist effectively has more pricing information than commercial providers of rental data do — and offers a more real-time look at the housing market than does the Census Bureau.

"We were looking for something more comprehensive, fresher in time, and smaller in spatial scale," said Geoff Boeing, a PhD candidate in the Department of City and Regional Planning at the University of California at Berkeley. "Craigslist seemed like an obvious candidate."

Boeing and Paul Waddell of the Urban Analytics Lab at Berkeley scraped millions of listings off the site from the summer of 2014. The data they sorted, described in new research — and mapped below — reveals some familiar patterns: New York, the Bay Area, Boston and energy-booming North Dakota have the highest median rents on offer in the country (in the map, red is the most expensive per square foot). And many of these same markets have a paltry share of listings at price points that would be affordable to moderate-income households.

1.5 million rental listings on Craigslist, mapped by cost per square foot. (G. Boeing and P. Waddell in the Journal of Planning Education and Research)

But the data also gives a fascinating look at the whole spectrum of offerings in each Craigslist market. In this graph from the research, each line represents a single metropolitan area, with its distribution of listings ranging from the cheapest at the left to the most expensive at right ($4 per square foot would be the equivalent of a 1,000-square-foot rental for $4,000 a month, which is not uncommon in New York). The lines peak at the most common per-square-foot price point in each area. As with the above map, the markets with the highest median rents are red; those with the lowest are blue and purple:

(Boeing and Waddell)

That picture shows that affordable cities have more compressed rental markets on Craigslist, while the distance between high- and low-end units in expensive cities is much wider. Detroit is narrow and spiky. New York is low and stretched out. Detroit's pricey units are not that pricey, and that segment of the market is much smaller.

Put another way: If you have a little extra money to spend on rent in Detroit, it will get you a lot more than in New York, bumping you from near the bottom to the top of the market more easily.

Become a CALIFORNIA REAL ESTATE AGENT 

In the Bay Area (which Craigslist defines much more broadly than just the city of San Francisco, encompassing San Jose, Santa Cruz, Oakland and outlying suburbs), there are hardly any units available at the per-square-foot prices that cover most of the Atlanta-area market:

(Boeing and Waddell)

Across all these places, the correlation is striking between the typical rent in a given market and the degree to which the market is compressed.

"We didn’t know what the pattern would look like," Boeing said. "We didn’t expect it to be so clear."

This pattern also illustrates why moderate-income households — and even middle-class ones — have such a hard time finding affordable units in expensive cities. There just isn't much on offer at cheaper prices. And this pattern implies that a subsidy like housing vouchers in low-cost cities may have a lot more power to lift the poor into higher-quality units and safer neighborhoods.

These pictures are not perfectly representative of the entire rental market in each region. Like census rental data, which lags in time, and commercial data, often drawn from large apartment buildings, Craigslist has its limits as a window into the housing market. It may exclude landlords uncomfortable with the Internet (or who believe their potential tenants might be). It captures only asking prices, not agreed-upon rents, so it doesn't reveal the effect of bidding wars that might drive up rents in high-cost cities.

And the quality of the data is better in some markets than others. The listings in Seattle and Los Angeles, Boeing and Waddell found, tend to have more complete information. In Chicago and New York, listings are more likely to be posted multiple times. New York's rental market is alsoheavily influenced by brokers, meaning units are less likely to wind up on Craigslist.

Boeing and Waddell originally scraped about 11 million listings off the site, covering everything posted in all the U.S. sub-domains between May and July of 2014. But by the time they deleted duplicate listings and inevitable Craigslist spam ("Apartment of $1!"), and sorted for only units with clear price and square footage data and geolocation, they were down to about 1.5 million listings nationwide.

That's still more comprehensive than many alternatives. And Craigslist has the potential to detect patterns other data sets can't, such as how quickly rents rise over time — in real time — in a single neighborhood.