Are tweets a good measure of patient experience and health care quality?

Twitter speech bubblesWhen it comes to gauging quality, we often turn to Twitter, Yelp, Angie’s List and other networks for instant feedback on pretty much any company, contractor or store we do business with.

In contrast, hospitals often rely on tools like the Hospital Consumer Assessment of Healthcare Providers and System (HCAHPS) survey, which gives voice to patients and their concerns about the care they receive. But a new study published in the journal BMJ Quality & Safety suggests social media have something to add to that.

Notes_title_for_overlay“The main problems with measuring patient experience by survey are the small numbers of people who respond to surveys and the lag time,” says Jared Hawkins, MMSc, PhD, of Boston Children’s Hospital’s Computational Health Informatics Program (CHIP). “It can take up to two years before survey data are released to the public. Given that social media data are close to real time, we wanted to see if we could capture this discussion and if the content is useful.”

Hawkins, with Boston Children’s chief innovation officer, John Brownstein, PhD, and their colleagues collected more than 400,000 public tweets directed at the Twitter handles of nearly 2,400 U.S. hospitals between 2012 and 2013. Using machine learning, natural language processing and manual curation, they tagged 34,735 patient experience tweets directed at 1,726 hospital-owned Twitter accounts. They classified the tweets as positive, neutral or negative in sentiment and binned them into topical categories (e.g., Time, Communication, Pain).

Their data were revealing. Tweets commented on patient discharge instructions, waiting times, hospital conditions and more. “We were able to capture what people were happy or mad about, in an unsolicited way,” says Hawkins. “No one else is looking at patient experience this way because surveys ask very targeted questions.”

Tweet sentiment did not correlate with HCAHPS experience data, perhaps because of the relatively small number of Twitter users in 2012, but the study did find a weak correlation between tweet sentiment and hospitals’ 30-day readmission rates. “Hospitals that people thought highly of had lower readmission rates,” Hawkins explains.

Read more on our sister blog, Notes.