Last year we wrote about a proposed system for rating hospitals with stars based on their HCAHPS scores and whether it would lead to the Michelinization of healthcare. The idea was severely criticized by the Association of American Medical Colleges and others.
But actually such star systems are already in place. For years the Centers for Medicare and Medicaid Services (CMS) has been evaluating Medicare Advantage Plans on a five-star scale. It also rates nursing homes in that way.
Not only that, but Yelp has been scoring hospitals on a five star system. A much-publicized Harvard Business School study reported that 16 percent of Yelp restaurant reviews (its best known category) are fakes. So one has to wonder about what they could do with hospitals, which really are much more difficult to evaluate
Accordingly, we took a look at the Yelp ratings for hospitals in the Boston area. The results were not impressive. Certainly the institutions that got the most stars were those with the best reputations (such as Massachusetts General Hospital); so one could claim that intuitively they made sense. However, the quality of the reviews is more than questionable. One person gave no stars to a clinic because he could get a timely appointment – a valid criticism no doubt but hardly an indicator of the care given there. Likewise, the five and four star reviews seem to be based on general knowledge and status rather actual experience and valid assessment.
So these ratings are all nonsense, not to be taken, seriously, right? Apparently not! As Yelp brags, some academics made a study of their ratings vs. HCAHPS and found a correlation.
Appearing in the March 2013 issue of BMJ Quality and Safety (“the international journal of healthcare improvement”), the four authors from the University of California at San Francisco “the correlation of percent high ratings between Yelp and HCAHPS was 0.49 (p<0.001),” which is a very strong by social science standards. Thus, they say, “the data suggest that the ratings posted on the commercial website may be capturing experiences similar to those driving the more systematically collected HCAHPS ratings, and that improvements in either patient experience measure may be associated with improvements in patient outcomes.” But couldn’t one argue that since the Yelp ratings are obviously meaningless, that correlation just proves that the HCAHPS ones aren’t valid either?
Regardless of all that and admitting that scholarship is necessary, beneficial and not to condemned no matter what the topic, there a question of what is the goal. Everyone says that our healthcare system is broken and needs to be fixed, but is chasing after numbers the way to do it?
As with so many social and organizational problems (education and testing being a prime example), emphasis on numbers can actually detract from – and be more attractive than – the hard work of actually changing things. An industry of “improve your HCAHPS scores” has arisen, but if all the money behind it had been put instead into actual healthcare advances (nurses’ training, staffing, and salaries, say) might not the benefits be greater?
Still, hospital leaders can be expected to keep their eyes on the stars. And here’s another thought for them: what about ratings for hospital food? Might not those correlate also with HCAHPS and Yelp? There must be money to be made in investigating that subject, too!
–Jim Murphy has a solo consulting practice called Management 3000, focusing on organizational development and change management. Formerly he led the Massachusetts Bay Organizational Development Learning Group, was Human Resources Director for the City of Boston Assessing Department, and served as a consultant with the Boston Management Consortium. His consulting practice includes management coaching as well as research and writing on employee relationships, leadership, healthcare and collaborative practices. Having produced newsletters for several organizations and being a frequent content writer for the”Confident Voices in Healthcare” blog, he is interested in writing and research opportunities, as we all consulting and coaching. www.manage2001.com firstname.lastname@example.org