4.9 Stars ≠ Good Care: The Problem with Healthcare Metrics (with Larry Benz)

0:00 No problem. And are we starting with the article? I don't remember what we just said. Yeah, I was going to do the article first. Yeah, perfect. Yeah. All right. Most health care businesses have a number they're proud of, a number they show off, a number they track, a number they think proves they're doing good work. Larry's argument in his latest article, that number might be completely meaningless. Today, we're talking about one of the most visible metrics in health care. Google reviews, five stars. Zero outcomes? Welcome to The Operator, a show about running better healthcare businesses, not from theory, from people actually doing the work. If you're responsible for patients, people, or performance, this is for you.

0:44 Larry, welcome to your show. Well, thank you, Jerry. Thank you for hosting my show. Well, my job is to sort of set you up. This is the last part in a series that you've been doing. And if people want to go back to it, we'll give the link to the sub stack that they can view as well as the articles on LinkedIn. But first, why did you decide to do sort of a series on metrics? What about that made you say, I'm going to dedicate several weeks of my writing to just metrics in general before we get into the specific Google reviews? Yeah. What we tend to do in an operating business and health care platforms is we tend to get very routinized on certain metrics that are like sacred cows. They're the belief that they are the most important ones.

1:27 We've had vast changes in our operating environment. If you think of sort of the post-COVID hangover days, then you had shortages and you had all the headwinds of reimbursement. and all the things around high inflation. And the business models have had to change. You've had to adjust. And so what we tend not to do in health care operations is rethink, challenge our assumptions. And say, why are we continuing to measure that? And then the objective becomes the measurement rather than the activities and the behaviors that really get the operations flowing in the right direction. And so I've tried to on each one of these metrics doesn't mean, as I point out in the articles, that you shouldn't be measuring them, but you should be understanding how they've shifted over the years.

2:15 And the whole area around good hearts is a bias that we tend to default in over time if we're not careful. Yeah. When the metric becomes the measure, it fails to measure what matters is the thing that kept sticking with me. That makes sense. I mean, you're Russian. If you want to go back and read it, the Russian nail factory sort of highlights that pretty well. When you tell me to gamify, when you gamify a metric and you put my salary on the on the line for that, guess what I get really good at hitting that number is the letter of the metric or the spirit of the metric. Well, I hit the letter of the metric and that's what gets me the paycheck. Just to that point, years ago, I'm going back into the late eighties.

2:56 I had a few clinics and one of my clinics, we used to measure no-show cancellation rate and used to incentivize the front desk person for a lower no-show cancellation rate. And interesting, she went on maternity leave and she was a wonderful intake coordinator receptionist, what we called DOFI at the time, Director of First Impressions. And all of a sudden, our no-show cancellation went sky high. And I started wondering, how is that the case? Well, as it turns out, you can game that number. It's no different than the Russian nail factory. Somebody cancels, you just take them off the schedule and put them somewhere else and you never were able to see what really... happen there. And so gamesmanship working around it, our tax laws are the absolute, you know, personification of good hearts law, everybody works around the tax laws to try and save on their taxes.

3:46 That's normal. Yeah. So we have to, you know, we have too many tax law issues, and good heart issues in operation. And so my challenge is somebody who's in the field every day operating is for us to challenge those assumptions. And you can tell which side of the argument someone is on with taxes when they call it a write-off or a loophole. If it's a loophole, you can tell they don't agree with it right there. All right, so let's jump into this. If someone logs on Google website, somewhere where ratings are being tabulated and shown off, what is a four point nine star four point out of five star rating actually tell you about specifically a health care business yeah honestly it tells you that they're good at asking that's not nothing not what most operators think it is you know four point nine means your patients felt well enough to leave a review that had a positive enough experience to give you close to a five star does not tell you about the patient who dropped off after the two to three visit cliff in physical therapy or in dental it doesn't tell you the patient who didn't show up for the second visit which oftentimes in dental is the most important visit from an economic standpoint it doesn't tell you about your twenty two day wait period for a new evaluation it doesn't tell you whether your functional outcomes improved or whether your hygiene appointment rate is collapsing It's a sentiment snapshot from a very self-directed audience.

5:11 And in health care, that audience skews very grateful, skews compliance, skews towards the people who finished care, the people who like you. The four point nine is real. It's just not the whole story. It might not even be the important or really key elements of the story. Yeah, you want to gamify that number. Don't point out where to scan the QR code to leave a review to somebody who's disgruntled, right? Although you can look at the other side of the argument, which is sometimes when people have a bad experience, they're more likely to go rate you and leave a review versus the people who are the silent, happy, satisfied customer or patient. Exactly. And it has some meaning. It's just not all that much important from the day-to-day operations.

5:54 Right. So at what point did healthcare cross the line from measuring experience to pretending experience equals quality? Yeah, so we gotta go back a little bit in place of time. In the, call it the late eighties, early nineties, there was this new thing that happened in healthcare called functional outcome measures, SF-II, the SF-XXVI, short form, long form, XXXVI, which is the healthcare quality of life. And so we started to say, what do outcomes mean? And outcomes were defined as days, dollars, utilization, functional outcome, And on top of that, we had patient satisfaction, later became patient experience.

6:35 And so the combination of these dollars are sort of utilization-driven data, clinical functional outcome-driven data, and all the things around the experience became sort of the three distinct domains. But what happened is over time, we've tried to simplify it. You know, we're in business and operations. We're always trying to make the complex simple from an operation standpoint. That's a great thing. However, sometimes when we make things so simple, we destroy what the animal that we're trying to create is. So the moment we ran out of better alternatives and stopped admitting it, clinical outcome measurements in the most outpatient settings is inconsistent at best, absent at worst.

7:16 Functional tools do exist, photo, promise, all the things are on Medicare, you know, PSFS, for example, but adoption is not, you know, is not evenly put. And at times, we tend to only do it on our best patients or those who complete care. And so you have to be really, really careful. And so what fills that volume, whatever is visible and Google stars are everywhere. And so let's use Google stars. I had series a couple of weeks ago about Net Promoter Score, a lot of similarities here. They're on your profile, they show up in the listing, they're a number, numbers feel like data. So we promote them, not because anyone made a deliberate decision about it, but that four point nine stars equals clinical excellence.

7:58 Somehow or another, we conflated that. But because the infrastructure for measuring actual clinical excellence wasn't there or wasn't fulsome or wasn't complete, the proxy became the point. And that's always how it happens. We make a proxy a conclusion, and Google stars are now that conclusion, unfortunately. Yeah. It's funny that you bring up four point nine stars. And I think I know why. To me, five feels too perfect. It feels like I can't trust that because it's nothing's perfect. So four point nine feels as close as you can get. Really good. But obviously there's some real in there. So it matters. Isn't that funny how that like we've even even on the receiving end said five's too good.

8:40 I don't trust it. A four point eight that four point nine that feels perfect. More authentic. Feels more authentic. Right. Right. So you mentioned like sort of the pre-COVID, right? So are operators still chasing Google reviews based on a version of the algorithm that doesn't actually exist? Are we still having a twenty sixteen mindset in twenty twenty six? No, that's absolutely right. A lot of them are. And that's really what my research started to show, that the old mental model was pretty simple. More five star reviews equals higher ranking equals more patients. So it was all driven by how high on the Google best search engine, period. It's interesting. We're switching now from search engine to AI, LLL.

9:25 And for a while, the volume of reviews was the dominant SEO signal. And this really is the main teaching point of the article. So that logic held true. But the algorithm has gotten way more sophisticated, as they always do. Technology always outpaces everything we do in health care. proximity, relevance, behavior signals, how people interact with your listing, whether they click, whether they call, these matter now as well. So it's the behavioral elements, not just the star elements. Review recency matters more than raw volume. So if you took a hundred reviews, it's like golf handicap system. It's the five top most recent ones that really matter.

10:06 So the operators who ran an aggressive, let's get to two hundred review campaigns. I still see this all the time. And they sort of stop. They're potentially losing ground to a competitor with eighty reviews. So that was kind of like I got two hundred. You got eighty. So therefore, I'm better. In fact, we are competitiveness. around google stars surpassed our competitiveness around clinical excellence and doing the right thing for the patient right you know the we got we got diverted by all this um so chasing the number at one time made sense now it's it's it's table stakes and the operators treating it like it's a competitive moat are you know probably optimizing for a version of local search that's already moved way way past that right you know by the way at least in physical therapy Why do we care about local surgery much?

10:53 Don't we have we have plenty of patients now, plenty of volume. Yeah. You know, really optimizing, as we talked about last week, optimizing that volume and not the churn is really, really important. So keeping the patients you have and what we're saying, milking your own cows first, you have those patients. Make sure you're getting the treatment completion on them so that you can focus attention on there. And that's where that, you know, really being extremely, extremely intentional about visit two and three and showing value there rather than routinize the visit. What you're basically saying to the patient is you don't need to come in anymore because we're going to do the same thing every time. Yep. I can actually hear someone creating a KPI just for that.

11:36 But isn't there one? Isn't it churn? So when you don't have real clinical outcome data, what does fill that gap? What is the right thing or the better thing to measure? Yeah, so what happens is we use Google Deck, Google Stars, and Net Promoter Scores because leadership needs something to say in a board deck. And this is a different topic for another time, but I think, you know, PE backed platforms and even private practices do not have the right type of board members. They have people playing by old rules of an old game, a, you know, call it college before we professionalize through revenue share and NIL.

12:17 That's the type of people that are on boards right now is this new game that took place post COVID. You show me an operator who's really done well the last few years and you've got an operator because they they're the ones who've dealt with incredible amounts of inflation at the supply level, inflation at the compensation level, shortages of your suppliers, all of those kinds of things. And so we use Google stars and net promoter scores for those board members because we want to fill a board deck. It sounds cynical. but it's accurate organizations under performance pressure from either PE owners, from franchise benchmarks from year over year need visible proof of quality. And so this proxy of Google stars is now what your compliance is showing up as quality.

13:03 And if you can't pull functional improvement data, very hard to do, as you know, If your outcome tool adoption is inconsistent, it's very hard to be consistent. If your discharge documentation is a mess, very hard to have that consistent as well because most patients don't read their end of treatment discharge. You reach the number you have. Google stars makes it easy. Net promoter scores make it easier. Patient satisfaction makes it easier. Those aren't bad metrics. They're just getting asked to carry weight they weren't built for. The danger isn't that operators track them. The danger is that over time, the proxy becomes the definition. You stop asking, are patients getting better? And start asking, are patients happy enough to leave a five-star review?

13:44 That's entirely different dimensions. They're related, but they're not the same questions. Yeah. You highlighted something in this article that I guess I didn't think about too much. And I bet you a lot of people didn't either, which is responding to one of these reviews, if it's negative or in a way, could open you up to liability. So why are healthcare providers the only businesses that really can't actually defend themselves in public? Or they can, but they have to be really, really careful about it. Yeah. So first of all, the reviews tend to reflect sort of your interpersonal and logistical experience as your wait times, your parking, the front desk, not clinical decision making.

14:25 And so the population leaving reviews is really not a representative population. people are complex they leave incomplete survey data disappointing outcomes are less likely to review stars of any any nature at all and what ends up happening is it's you know do you like your provider all those kind of things they review it but that selection bias is baked into every star rating in the industry But the biggest thing is what you're referring to on the question, which has HIPAA in the background, which limits how providers can respond to negative reviews. You can't get into details. You can't correct the record. You can't explain context.

15:07 So one-star review about a billing dispute sits on your profile essentially unchallenged because your response has to be generic. It lands and people people ignore that. And so now we've developed this whole niche industry about how to respond to bad, bad reviews. It's almost like all of us need our own PR crisis when we get a bad review. We also have to remember, you know, I always tell my therapist this and the PT side and I say the same thing in the dental. If a patient left your local clinic. because they refused to accept some level of self-responsibility in physical therapy that manifests itself by a home program or compliance with instructions.

15:48 And in dental, it manifests itself with compliance with instructions on hygiene care and other aspects of post procedural care. And they would not take and therefore they disliked you because they wanted to put all their problems on to you. It's OK to fire a customer every once in a while. And if that leaves you a bad review, you could still sleep at night because self-responsibility and an active role in your own health care, your own recovery and in your own in the dental world and your own treatment is critical. A lot of patients don't want to assume any responsibility and they might write you a bad review. You can't get overly intrusive on it because you're violent.

16:28 You could be violating HIPAA. Yeah, it's the juice versus squeeze ratio. You might chase after and put so much effort in this one patient, you're probably never gonna make that, you might never make that person happy. Let's take that juice versus squeeze ratio over to this question, because this takes time, effort, money. There are firms who you can hire that are specialized in this. So what are people actually spending, even if you're not hiring a firm, you're also spending brainpower, time, staff time, effort, money. What are we actually spending to maintain that And has anyone proven that it does anything? Does it drive business or patient growth at all? Yeah.

17:08 So one of the questions I ask operators to think about is the concept of what is your return on the cost of quality? The cost is the time, effort, resources, money, all these things we do to ensure some level of quality. And so you have to be cognizant of that. The spending is extremely real and it's often invisible in the budget. You know, people aren't you're not budgeting this as much staff time training and review technology platforms with automatic post visit review sequences. Now we have all this stuff that goes into your phone that automatically asks you all of these kinds of things.

17:49 And for a multi-site operator, man, that adds up. fast and keep in mind all the SAS software providers, how do they make more money? The same customers adding additional APIs that allow you to do more functions is how they make more money. You know, we'll talk about the EMRs and other kinds of things later because the LZRs miniature versions. So now I'm not only doing your EMR, I'm doing your point of service pay at times. I'm doing your outcome studies. I'm doing your documentation AI. All of these things keep adding costs. And so we say as operators, where's my cost of RCM? Should be going down. All of a sudden it's going up. Why is it going up? Because all these providers are adding APIs.

18:30 They're increasing your cost and you have to do the tough decisions and make choices. about all of those kinds of things. And so in health care, a lot of choice is constrained. Insurance networks, referral patterns, geography, they're still doing an incredible amount of routing. And so if you're investing to maintain a four point nine in a market where a meaningful portion of your new patients are coming through other referral channels that aren't looking at your Google rating at all, that doesn't mean the investment is wasted. It just means that ROI calculation is messy. They're the platform selling requests. So we see all of these efforts now in the AI world of lead generation and all of these things around it.

19:13 And I talk to practices and they say, I'm not having trouble getting new patients right now. Well, you're spending an incredible amount between your foot marketers, your advertising, your digital marketing, and all of these things have accreted to a level. And you have to say, does the business model support it anymore? And in healthcare, it's a penny nickel business, a nickel dime business. You can't afford to spend an enormous amount of money on a problem that you think is there, but it really isn't. Yeah. All right. So I'm going to hold your feet to the fire. I actually put your article into test. I said, hey, how would some leading brands actually optimize this? And one of the best responses was Disney.

19:56 They said Walt Disney himself would optimize for miles per hour, not miles per hour, not stars. So if Google stars don't tell the truth, what should operators be measuring instead? Yeah, so we'll get into this a little bit, but I noticed one of the comments from the article last week was somebody saying the best metric they measure is how many smiles are leaving their clinic. And I thought that was a real interesting one. In dental, at least in our company, we say our mission is to create a lifetime of healthy smiles. And how are we doing that? Doing meaningful and purposeful work and remembering. In fact, that's what we're here for.

20:36 But there's a short list with teeth. New patient arrival rate is a real critical one. Are new patients choosing you? Is that number growing? Is your number of new patients growing? We talk about growth in organic and all these things and platforms, and most PT operators measure that wrong. They measure growth in terms of number of visits per patient, all kinds of things. They ought to be measuring it by growth and cash flow. Different topic for another time. But is the number growing? That's a market signal, not just a sentimental drop. PlanetCare completion, very, very difficult, but really, really, really difficult. Are patients staying through their episode of completion?

21:17 To me, that would be the best proxy. of a patient you know satisfaction really because dropout is the quiet killer that a four point nine doesn't capture right functional outcome improvement discipline specific systematically collected not cherry-picked reactivation rate are former patients coming back when they have a new need i call it friends family and former patient percentage what percentage of your new patients fall into that bucket that is really critical to me because it means you're spending zero dollars under what you should be doing to get your marketing, which is providing great care. That's continuity and it's its most measurable form.

21:57 None of these are as clean or as visible as a star rating. You know, in order to get to the answer, it's not a simple answer like a star rating. That's exactly why they matter more. The metrics that are easy to show off are usually the ones that have already been gained. The ones that are harder to show off are the ones that's actually happening. Yeah. Anything you'd want to leave the audience with as we wrap up? Your parting thoughts? No, just, you know, not telling you not to do Google ratings. Just don't overvalue them and don't spend all you're trying to get more of them. Reroute that same effort into something that brings meaningful change into the operations of your practice. I mean, as a guy who's spent some years in marketing and communications, the best marketing is a really, really great product and or service.

22:42 If you're trying to become a better operator in healthcare, you're in the right place. If this episode hit, Larry goes deeper inside his writing. Look for the operator on Substack. We'll also drop a link below. That's where the full breakdown lives. We'll see you next time.