Yelp ratings of healthcare facilities may reveal death rate disparities

“Much of the focus in healthcare is on quality and outcomes,” she said. “Patient experience is also critically important and should be factored into how to improve care across the board. This appears to be one novel data source for doing that.”

More than 95,000 facilities that provided some form of care recognized by the Affordable Care Act were included in the study led by Merchant and its lead author, Daniel Stokes, MD, a researcher with the Center for Digital Health and an internal medicine resident at UCLA Health.

Each facility included in the study had at least three reviews between 2015 and 2019 on Yelp. Each healthcare facility’s ratings were also coded to the specific U.S. where it was located, resulting in more than 1,300 counties—roughly a third of the country—being represented in the study.

Overall, healthcare facilities achieved an average 2.9 score out of 5 stars, but reviews were weighted very heavily to either side of the scale: five-star reviews account for 52.9% of all reviews, while one-stars made up 33.3%.

But when researchers looked at the county-level data of reviews, they found that five-star reviews within the group with the lowest death rates made up 55.6 percent of their total, while one-star reviews were at just 29.1 percent. In the group of counties with the highest death rates, five-star reviews made up only 42.9 percent of the total, compared to 38.8 percent one-stars.

The researchers then determined that if a county’s health facilities’ reviews were a star higher than their average—one point on the scale—models indicated that it translated to 18 fewer deaths per 100,000 residents. But when the study was refined to include counties with three or more healthcare facilities, the impact was greater, indicating a reduction in roughly 53 deaths per 100,000. Refined even further to counties with five healthcare facilities or more, the impact grew to about 60 preventable deaths.

Yelp reviews provide narratives, which was particularly useful to the study. The researchers used natural language processing algorithms to gain special, categorical insights. They showed that the types of words most associated with one-star reviews related to time [such as “hours” and “waiting”], payment [“money” and “pay”] and interpersonal interactions [“rude” and “told”].

Common language in five-star reviews changed depending on location. In high-mortality counties, “friendly,” “nice,” and “staff” were all typical, while low-mortality counties were associated with “Dr.,” “helpful,” “question,” and “pain.”

The researchers believe their work, published this week in the JAMA Network Open, adds evidence that unfiltered online repositories like review sites and social media contain valuable patient feedback and are an untapped resource for informing healthcare providers about what they do.

“Online reviews of healthcare facilities provide direct insight into patients’ experiences of care and can be a powerful force in shaping the care we provide to be more patient-centered,” Stokes said. “This has important implications for both individual and community health.”