So What? Interpreting Customer Satisfaction Score Data

So What? Marketing Analytics and Insights Live

airs every Thursday at 1 pm EST.

You can watch on YouTube Live. Be sure to subscribe and follow so you never miss an episode!

In this week’s episode of So What? we focus on Interpreting Customer Satisfaction Score Data. We walk through what customer satisfaction scores like NPS scores are, how to compute your own satisfaction scores with tools like Google Analytics 4 and what do to with customer satisfaction score data and how to apply it. Catch the replay here:

 

In this episode you’ll learn: 

• What customer satisfaction scores like NPS scores are
• How to compute your own satisfaction scores with tools like Google Analytics 4
• What do to with customer satisfaction score data and how to apply it

Upcoming Episodes:

• TBD

Have a question or topic you’d like to see us cover? Reach out here: https://www.trustinsights.ai/resources/so-what-the-marketing-analytics-and-insights-show/

AI-Generated Transcript:

Katie Robbert 0:25
Well, hey, Howdy everyone. Welcome to so what the marketing analytics and insights live show I feel like my voice went like way up for some reason. But I’m back to my normal speaking voice. I’m Katie joined by Chris and John. Definitely trying to stay focused today. I don’t know where I am. On today’s episode, we are talking about interpreting customer satisfaction score data, sometimes also called NPS scores. Chris, what does NPS stand for?

Christopher Penn 0:54
Net Promoter Score? It is a methodology for calculating. Well, yeah, I have a whole thing. We can walk through on that. But I’ll let you get the introduction first.

Katie Robbert 1:05
All right, thank you. I’m definitely like my brain is Ping Pong today. So I apologize for that in advance. So we’ll be covering what customer satisfaction scores like NPS scores are, what the heck they are. This is a question I asked you all the time, Chris, how to compute your own satisfaction scores with tools like Google Analytics 4? And what the heck to do with it? That’s the other question I asked you all the time is great. What do I do with this information? And so we’ll be covering all of those pieces today. So Chris, let’s start at the top. What is a customer satisfaction score? What is an NPS score?

Christopher Penn 1:39
Okay, so a customer satisfaction score is pretty simple. It’s how satisfied your customers are. And there’s so many ways to do that. Right? You can ask people, did you like this thing? Have you ever been to the like the airport or, or some restaurants and stuff, there’s like that little table, you can hit like the smiley face button or the frowny face button, depending on how your visit was this? There’s all sorts of different ways to measure it. One of the methodologies that is really well known in business is called the Net Promoter Score, or NPS and NPS is a computation created. I think it’s Nielsen, I don’t quote me on that, I’d have to go look it up. But essentially, it asks people on a scale of one to 10, how satisfied are you with the thing? Now, here’s what’s different. The math that they use, is kind of interesting. So on that scale of one to 10, anything below a six, they consider a detractor. These are people like, oh, yeah, you know, it was okay. It’s not great, or it sucked. I didn’t like the sandwich or the cheeseburger or whatever. Seven and eight are considered neutral. And then nine and 10 of the promoters, people who are like, Yeah, this is awesome. I’m gonna go tell a friend. I really enjoyed it. And so what NPS score does is it essentially takes the number of promoters minus the number of detractors does a percentage basis, it says that is your Net Promoter Scores, if you have, and the reason that they’re logic behind this is that you would think that would be or could be evenly balanced, but they’re like, Yeah, mediocre, is still bad, mediocre, is still a detraction from, from what’s good. And so that’s the sort of the basis of their their scoring capacity. So that’s what a Net Promoter Scores is a tabulation. In fact, there’s, there’s even math for it, on how to compute what those scores are, as you know, it’s the percent lack of a percent likelihood. So that’s just doing the percentages of percent, like there’s likely to vibratome percent unlikely divided by total. So in the NPS world, its promoters divided by the total, minus the detractors divided by total is your Net Promoter Score.

Katie Robbert 3:49
I asked about this in our free slack group this morning. So trust insights.ai/analytics, for marketers, and what our members thought of Net Promoter Scores, and someone one of our members made the various do comment that, you know, what I think is a seven, someone else is going to think as a 10. And when I look at this range, and how wide it is, I can see where this is problematic. Because, you know, as someone who’s done survey design, there’s way too, it’s the scale is too broad, there’s too many options, and it needs to be so a typical Likert scale sometimes is a one to five and so one being not at all, five being extremely, three being, I’m neutral, and then two and four being the I’m so so not but um, so So am, you know, that’s a fairly straightforward Likert scale, this scale. When I see this, there’s too many options. There’s, you know, what is the difference between a three and a four? What is the difference between a SEVEN and an EIGHT really nothing.

Christopher Penn 4:56
There really is nothing except in the back end mathematics and so This is the problem with Net Promoter Scores. And one of the criticism of them is that they don’t take into account how people behave. So at Trust Insights, we use a modified version of this, and you will see this in our newsletters and on our website stuff. Ours is like this. Thumbs down, thumbs up or don’t care. And it’s just three options. Because if you look in the back end data, very often, when people are doing scores, just like Amazon reviews, you get a one star, you get a five star, you don’t get anything in between, right? There’s always like that one goes, a guy does actually have star because of this thing of like, yeah, okay, whatever. You have too much time on your hands. But for the most part, people are very thumbs up, thumbs down, or I don’t care. And those to us seem to be the easiest way to simplify the mental load for people on responding to a survey.

Katie Robbert 5:52
John, what do you think of Net Promoter Scores? Have you ever worked at a company where Net Promoter Scores were, you know, a big measurement of success?

John Wall 6:01
Yeah, I mean, I’ve gone so far to is, you know, companies that are actually taking financial compensation and stuff like that to NPS. So it’s, yeah, it’s crazy the way it goes, I mean, really, you know, this approach here is the same psychology behind the other scale, really, like when you look at the one to 10 scale, the idea is that you’re only looking at three possible votes, it’s, yes, I love you guys. And I’m going to talk to other people, or, you know, yellow is the seven, eight, where it’s like, it’s okay, but it’s not enough to talk about and then yeah, everybody below six has a problem, right? There’s an issue, there’s something wrong. And so the the one to 10 scale, that’s gives you some extra great granularity by spreading out you do kind of figure out how pissed off people are, because that’s the one thing that’s interesting, if somebody is, you know, even just willing to talk to their neighbor, versus the fan that’s out promoting you, that’s really just a plus, like, those are all just plus ones. Whereas somebody who’s angry with you, at a five or six is different from a person who’s a two who could literally drive hundreds of customers away or drive you out of business, you know, the, the impact is negative in there. And then the other thing that’s with all of this, is that it depends on how you use it, you know, if you’re going to use it to generate more customer discussions, and actually drive into find out why you got bad scores, or why you got good scores, it’s great. If you’re gonna run 5000 surveys, and just people are gonna stare at the number and have board meetings about it, then that’s the system right for gaming, and you know, it becomes a whole nother a whole different thing, which is, it’s, if you have no other measurement stick, then that’s great. But you can do better too, if you want to, it’s just a matter of, you know, what you want to do with it, how much gaming you’re willing to put up with around it, if it gets big.

Christopher Penn 7:47
It’s like every Uber driver was saying, Please give me your five stars.

Katie Robbert 7:52
Yeah, I feel like, I feel like online reviews is a whole different topic that we should cover at another time. We were actually debating the merits of those reviews this morning, as well. But this is essentially, you know, a review system for a company or a service. So that’s right. What’s next, Chris? What do we what do we need to know about NPS scores?

Christopher Penn 8:14
Well, let’s talk about what do you how do you how do you get this information? Because it is it would seem like pretty straightforward, and yet, it’s something people struggle with. So in the Trust Insights newsletter, every month, we run this poll is the one click poll. And our measurement, our data collection mechanism for this poll, is Google Analytics, right? Because we want it to be something that simple, will have to be something where we can get at the data. And we want to be something that we can process and other people can look at it they want us to, but we could certainly programmatically get at it. And so what you’ll notice is, there’s this poll, and this three buttons, likely neither likely or unlikely and unlikely. And for next month’s poll, I am adding emoji.

Katie Robbert 8:59
Oh, to be fair, I did say you could so

Christopher Penn 9:01
yes, yeah. And in the back end, all three of these options go to the same page on our website. What’s different is that there’s a query parameter that use as NPS poll answer likely, NPS poll answered nyvac, or NPS poll unlikely in the question that we asked is this, how likely are you to recommend Trust Insights as consulting firm to someone in the next 90 days? Because this is essentially, it’s one of the best questions to ask and surveying for just general satisfaction, because if someone’s not likely to recommend you, they either don’t know anything about you, or they really don’t like you, right? You could also ask questions like how satisfied were you with the service that you received today on today’s plumbing visit or whatever? But I like this predictive question because there’s a difference between how satisfied am and whether or not I’m going to promote you. I’m going to I’m going to recommend you and so I like this question as a promotion question.

Katie Robbert 10:02
It’s, it’s interesting, because, as you were saying that, you know, and this is the question that I that we’ll get to have like, what the heck do you do with it? One of the things always I’ve always struggled with is, I don’t understand how this information is helpful. But now as we’re talking it through, which is, you know, admittedly, probably something we should have done a while ago, or maybe before this show. Because not everyone who subscribes to our newsletter is a client. However, what you’re asking is, are we doing a good enough job of explaining our expertise and what we do? So that if somebody said, Hey, do you know anyone who can fix my Google Analytics, or, you know, come and consult on a large learning model project? Then you’ve read our newsletter and said, Yeah, I think Trust Insights does that that’s what they’re always writing about. So now, it’s like the, the pieces are starting to click for me. But, you know, I still need more information. But this is, okay. I see where you go on with us.

Christopher Penn 11:01
Yep, exactly. And the advantage is using this style collection, because these are just query parameters inside of a URL. A, there’s no, you’re you’re just using Google Analytics as the collection mechanism. And the answers are stored within GA, they’re stored on a time based format. And they require no extra work. So if I go into our Google Analytics account, I go into explorer, I just start with the page path and query string, which captures all the parameters and the number of events. And then I say, I want to filter it based on that NPS poll answer, because you remember, that’s what we use is the query parameter in the URL. And now I could see, neither likely unlikely, and then there’s a couple of anomalies. So we can see, like, neither is this is for the last 13 months, neither is 567. Likely as 560. Unlikely is 166. So, to me, that’s the just that alone, knowing that alone, is that okay? We’re not like, I would even be worried if it was evenly distributed. Something’s wrong with evenly distribute, but the fact that neither or likely are way ahead of unlikely makes me feel better.

Katie Robbert 12:16
Yeah, my first thought was, oh, people are neutral about us. But also, yay, people are neutral. Like, I think it’s okay. In business, if you have an audience who they’re kind of mad about you, because not everything you do has to be a viral thing or solve everyone’s problem. But you’re still retaining that audience, because at some point, something you do is going to resonate. So yeah, my initial reaction was oh, man, but now more, the more we dig into it, it’s like, okay, that’s not that bad. Because they could be swayed either way. So now it’s on us to make sure we move them in the right direction.

Christopher Penn 12:57
Exactly. And to your point earlier, the composition of the audience matters, right? So if we have, for example, a lot of college students, I’m just giving this up if we had a lot of college students on our mailing list reading this, would they recommend us to a colleague, probably not because they don’t have anyone to recommend us to in that sense, like, I would have a junior at Wheaton College, say like, I’m going to recommend you to my colleagues as a consulting firm like that. So they’re probably in that case, answer neither likely or unlikely, because it’s not that they don’t like us. It’s just they don’t have anyone to recommend us to.

Katie Robbert 13:32
John, do you include any sort of satisfaction survey after marketing over coffee episodes? Like, did you like the episode? Did you?

John Wall 13:43
Yeah, up or down? No, no, we don’t do any tracking. With the time delay, it’s not like I would do any follow anyways. It’s a it’s good to get into a big part of it for us is, you know, you know, when you’ve done a good one, because you see more chatter on social media, people talk about it, you know, and just nobody talks about the bad ones or passes them on. But yeah, I don’t know, for podcasting. There’s not much as far as tracking and, you know, kind of, at the individual level, it’s more about, you know, do you trust the brand? And do they keep you entertained? You know, long enough? Over time? Are they good enough to hold your interest? Which is really where it’s at? Yeah, I don’t know, I probably should. The the more interesting one is NPS for sponsors. You know, are you happy with the leads that you’re getting and the value you’re getting out of spoiling the audience? And, you know, do you have leads that come in that say that they found out about it through us? And yeah, thankfully, we have a small enough audience that I’m able to get that kind of information personally, you know, that becomes part of two months after every package go in and ask how things went and how many leads they saw and how things turned out. And yeah, it verifies the results. So you know, you can tell at the end of the campaign, if it was somebody that has nothing to do you know, the Folks selling shaving materials to 50%. Female Audience. Yeah, maybe that was not the best use of your money. Not surprising the results, you get the wisdom that wraps up.

Christopher Penn 15:13
Yeah. Now there are other folks who are using these systems and using them very well. So here’s one this newsletter called The Rundown. And you know, is this this is a daily newsletter. And they asked How is today’s newsletter feedback helps improve nailed it average epic fail again, thumbs up, thumbs down. And okay. This is this is if you have a publication that you want to assess, this is a great way to do it. Because not only do you get that, that frequency of up and down and things, but you can now you know, the topics that were in that newsletter, or the news or the information we’re sharing, you can get a sense of these articles are these topics, they these are the ones that people care about, you know, there’s a big you nailed it that day. Other days, you know, maybe it was a thing on politics, it was, nope, I don’t want you anywhere in politics. And so you’ll get an epic fail for your audience. So these tools are incredibly useful. And they follow the exact same format that we do. So just click on the URL, that’s your vote, no extra work. And it allows rapid, rapid and good data collection. Like I said, this is right inside Google Analytics. So if you just want to do just the basics, let’s look at the last 30 days. Just the basics alone, we have 38 like least 27 knives and no unlikeliness, which is pretty darn cool.

Katie Robbert 16:37
Don’t take the wind.

Christopher Penn 16:38
Exactly. Take the wind. Yeah. So now, if you want to kick it up a notch, which I recommend you do, you want to take this data out of Google Analytics, and put it into a format where you can get a better understanding of what’s happening, particularly over time. So this is the exact same data. But it now uses the NPS score format. Promoters divided by the total, minus the detractors by the divide by the total equals your NPS score. So for us, the green line represents point five or above point five or above is where you want to be the red line, which is zero, you do not want to be there. And so this is our last almost full year of this data, there’s a minimum of 25 responses per month, if a month doesn’t get 25 responses, it just won’t show up. Because we didn’t want to have like two people responding. But thankfully, that doesn’t happen. And you can see, since about April of this year, we’ve been above the green line more than we’ve been below it. And April was a really good month, last month was a good month, the month before was a good month. So our our our NPS scores for our, from our newsletter, to our audience are generally very positive. And this is a this is a big change from last year.

Katie Robbert 17:54
Well, I’m trying to figure out John, who to who did you upset in February that we got such low scores?

John Wall 18:00
I know is hopefully not tied to Valentine’s Day in any way. That’s my concern.

Katie Robbert 18:05
But no, I mean, it is it’s, you know, I look at this, and I’m like, That’s great. We’re up. But my you can imagine, Chris, so my first question is going to be so what? Why, why? Or what? What’s the difference? How, like, how do we know what we’re doing? Well, what changed? You know? And so there’s a lot to unpack when you see a very simplistic chart like this, like you might be thinking, Oh, this, I can present a very simple chart. And people will understand it. Yeah, I see that we’re up. But yeah, exactly. So what, what do we do about it? How do we know what’s working like, and then the million questions start to come. So where do we start to unpack that information?

Christopher Penn 18:45
So another useful thing about doing this surveying with your email newsletters having that in there is that if your email marketing software is any good at all, you get individual click tracking. So you could go back into your back end database for your software, and say, for the last month, show me which of the 68 people click likely. And you can go back to those 68 people and email them and say, Hey, I noticed you voted likely. Why? What What made you likely what your what did we do that? You said, Yeah, I want to recommend Trust Insights to a to a colleague, you can go to the zero people that were unlikely to say like, what do we do? What are we not delivering to you that that would help persuade you? And you go to those the neither likely or unlikely and say, Okay, well, what else could we do? That would change your mind? If anything to go from not? Yeah, I don’t know if I’m gonna recommend you or not to Yeah, I’m going to recommend you. So there’s because you have that data at the individual level. Obviously, we can’t show it on screen because it’s PII. You can go follow up with people and say, What can we do better? Or what did you really like?

Katie Robbert 19:55
I would imagine, well, not even I can imagine I know for sure for a fact that there’s a way to automate that feedback process. And so I’ve given feedback to other brands that I’ve gotten, like customer service with or, you know, it’s a lot of times where it’s used is, you know, how satisfied were you with your customer service. And if you engage with the survey, see, like, you know, I would likely recommended not likely to recommend, a lot of times, you’ll get an automated notice that says, are you willing to answer a few more questions. And that’s a way to get to that detail, without having to take that manual step of giving John a list of people to go harass, and then our NPS scores go down. Because they’re like, oh, Chief statisticians after me. Now, what does he wants? He wants my money. He wants my time,

John Wall 20:41
more cold calls? Maybe.

Katie Robbert 20:45
But yeah, automating that feedback seems like a really smart way to approach it. And you’re right, any decent marketing automation tool should be able to help you do that, or a CRM system could absolutely do that.

Christopher Penn 21:00
You can even do it right inside Gravity Forms, right? Because again, we know who the person is, we have their, their email address. So what they could do is they could click through, and then end up on a page like this. It says, Hey, we got your answer. Do you want to tell us anything else? Right. So and I think

Katie Robbert 21:14
that that’s an excellent way to approach it.

Christopher Penn 21:19
Exactly. So that’s what you can do to get more granularity in the data to and to get that qualitative feedback that can can give you some direction with with the recommendation process itself, or it and it may reveal to you, for example, in the feedback, yeah, I don’t know who I could recommend you to. In which case, it’s not a question of, do we do something wrong, it’s that we didn’t make clear, here’s who you should be recommending us to.

Katie Robbert 21:47
Here’s the type of businesses that we serve.

Christopher Penn 21:51
Exactly. Make sense. The other thing that, and this is what I typically think of when I use this data, and I look at this data is I see this to see the intent to recommend is higher in some months than others, the intent to recommend is, is above the green line. To me, that tells me that someone like John could be more aggressive in sales pitches to say like, yeah, you know, what, people are generally positive about us, people are generally thinking about recommending us. That’s a good sign as long as statistically valid. I’d be okay with two sales promos this month instead of one, or I’d be okay with John doing some more. slightly harder selling or hitting more of the list of folks like yeah, you know, this person just didn’t demonstrate clear intent. But when that number is high, so you know, what they might not be, they might be okay with hearing from John’s, even if they didn’t raise their hand, say, I want to talk to John, John. So when you look at February, like John, Don’t call anyone this month

John Wall 22:56
is using this to drill down and see who’s who like, that’s a huge part of it. Because, you know, months, like this month, where we’ve given away a lot of top notch free resources. I mean, there’s a bunch of folks that we’re hearing from that are at universities, and you know, single person shops and stuff like that. So really, you know, digging down in to see who’s who is huge, because even even on a bad month, if I if there’s a fortune 500 CEO that rated us a nine, you know, I want to see what’s going on there. Exactly.

Katie Robbert 23:28
I wonder, too, and I’m, I’m assuming if we have enough data, we could do this kind of, you know, matching, I don’t necessarily want to call it a correlation. I don’t know if there’s enough, but, you know, do the peaks and valleys of the NPS score correlate with like, was it budget seasons? Or, you know, anything else? Like, is there some kind of seasonality to the NPS scores where we could be like, Oh, well, we know, you know, every April, our NPS score goes up. Because for some industries, that’s the end of their, you know, financial year. So they’re looking for new partners, versus February, where the new year just started. And everybody’s already burnt out from trying to figure out what the heck to do with their fancy new budgets, and some companies have already blown through 70% of it, and it’s only February. So no, they’re not going to recommend us to anyone. So when I look at this, that’s what I would be curious to know is, does it matches up with any other kind of data that would help us understand not only what are we doing well, and what are we doing wrong, but there’s a seasonality to it. So you know, in the example of like, giving John you know, either like hit the gas or pump the brakes, knowing ahead of time like we know April’s going to be a big month like we can look at our our own sales they didn’t know like, what months. So that would be interesting to to see like, does this correlate with our are better months than, you know, just looking at this by itself.

Christopher Penn 25:05
And that’s exactly right. However, we only started doing this in a reliable, repeatable format in October of last year, we’d done it sporadically in the past, but it was never something we were disciplined about until last year, like, we just need to start rolling this come hell or high water every single month first newsletter every month, this goes in in the newsletter. And we have not missed any since October of last year. So we would need probably at least another year’s worth of data just to to be on the safe side. However, we also do have data that’s that’s quantity of survey data with every form filled, right, we asked people, what are the things that you you’re interested in for better measurement, better tools, understanding ROI and things. And so we can certainly take this data and do that cross correlation analysis with that, do it with a website data, do it with social media engagement data, do it with a number of opportunities or contacts and Hubspot. So there’s so many things that you could do that that multiple correlation analysis to see what of those things has a correlation to this, this data? And then that gives you the starting ingredients to do okay, well, now, how do we prove any causality?

Katie Robbert 26:22
And I think that, that, at least for me, that’s the kind of when you say, like the so what, that’s where my brain goes immediately, like, Well, does this align with, you know, budget planning? Does this align with our sales data? What else could it potentially lead with? So like you mentioned social media data? I don’t know that. For me, that would really tell me anything. Or I could really do much with that. Because social isn’t a high performing channel for us. I would be interested to know like, number of net new email subscribers, month over month, because since we’re sending this out over email, and email is one of our stronger channels, does the number of net new email subscribers correlate to how high the NPS scores go? Because they’re people who’ve never heard of us before? Or they’re brand new, and they’re super excited. And then they kind of like, you know, we fall into the background noise after a few months.

Christopher Penn 27:21
Yep, no, I those are all valid questions that the data can help answer. It’s just a question of, Do you have the time to to put that together? And now, the good news is these days? The answer is more and more is yes, Katie, you and I were talking about this, in our company slack not to go about using tools like ChatGPT, use advanced data analysis to be able to do those analyses. Because even if you are someone who can’t do the mathematics of the coding, with a tool like that, you can give it the data and say I know what I want, I can put it into words, I just can’t write the code, it will write the code, and then you can get your answers.

Katie Robbert 28:05
So okay, so we’ve been talking a little bit about like what we do, but if you were to hand this data to an executive, what kind of an action plan would you put together? For them? What what actually do they do? Because we’ve been kind of circling around some ideas, but what should someone be doing with this data consistently?

Christopher Penn 28:27
to John’s point, most executives just either give themselves a bonus. Or just bury it quietly on the shelf,

John Wall 28:34
right? Or fire the customer service VP, right, the tuition staff change over there.

Christopher Penn 28:42
Now, the next thing to do is that qualitative research, you ask people, okay, why? Why would you recommend us? Why would you not recommend us? And, again, it gets back into like, you set to be programmatically integrated into the system, which is something we’re going to look into, because it seems like a pretty straightforward thing to do. And then, again, from from a sentiment perspective, being able to look at the data and say, Okay, John, here’s why I think it’s safe to turn up the heat. Or maybe it’s not, maybe let’s let’s deal with the inbounds to come in, but but don’t be chasing people down. So those are the things that I would say belong and an action plan is you have this intent, you can rely on that intent to make decisions to to do more or less of something. If this was customer service satisfaction data from like a call center. Then you say, Okay, well clearly, we need to be more carefully measuring our call center representatives as to the as the individual satisfaction that we collect after each call. JetBlue, for example, uses this exact system to rate every leg of a flight and they look at flights and say okay, wow, something really went wrong from this New York City to LAX flight that everyone waited two to one and that they can go dig into that and say Okay, well, here’s, here’s what happened. I was like, Oh, it was two and a half hours late leaving the gate, and the rest of us didn’t work. And so there’s some explanation there. But the key to this data is to use the data in any fact, don’t just take the chart, like do something with it.

Katie Robbert 30:17
I was gonna, I was going to mention that is that yeah, it’s great to collect the data. But that, that’s sort of where people get lost. And that’s whole, that’s the whole basis of Trust Insights is we light up dark data, don’t let this become another piece of dark data that just sits on your shelf that you collect, you don’t know anything about because you start to get into that, I guess, essentially, data debt of you have all these pieces, and then you have to go back and sift through it and clean it up and figure out, and then, you know, you could find that the majority of people who said they would recommend you are no longer part of your ecosystem, they’re not subscribers, they’re not your audience. So you can’t go back to them and be like, Oh, well, remember, like five years ago, when you said you were going to recommend us to someone, would you still do that, because they’re gonna find that pretty annoying and creepy. And then John’s gonna get really low NPS scores all over again, it’s a vicious cycle.

John Wall 31:13
You know, this is a great way for slicing data, though, when you’ve got a problem, because the one thing you can do is you do the first cut. And then you can take everybody seven and below and send them an email saying, Okay, we realize you’re really angry with us and have a problem. And you give them the five key areas where you normally have trouble with. And so now you’ve got them sliced again, and you can, you know, get to the core of what’s going on, like the classic Amazon reviews, right? Was it because it was damaged in shipping, I always love to see that people, you know, a product getting a one star because it came to my house crushed like, not that the company has anything to do with that. But you can kind of weed out the basically the folks that are just off their Rocker from the ones that have legitimate complaints. And at the whole time, you’re not bothering anybody that’s happy or content, you’re only working with the group that needs some attention.

Christopher Penn 32:02
And that’s a really important point for customer satisfaction. Because as we saw at the dawn of public social media, different channels get different satisfaction levels, right, your your Twitter call center versus your email call center versus your telephone call center all may have very different satisfaction scores. And they should they should all be equal or close to equal because it means you’re delivering the same level of service. And if you’re not, then you’ve got some training and education to do to help the call centers reach the same level of hopefully improving satisfaction.

Katie Robbert 32:40
When we were working at the agency, Chris, my recollection and correct me if I’m wrong, is that so I know that that’s where I was first introduced to NPS scores. I’d never heard of him before, prior to joining a public facing agency. That was a score that they would then also promote publicly of like, look how great our NPS scores. This is why you want to work with us. Is that a common practice for a lot of companies to say like, look, we have a really high NPS score. So we’re the ones you want to work with.

Christopher Penn 33:18
I mean, if you saw a job ad, that was advertising that how would that make you feel?

Katie Robbert 33:25
Oh, I would think that was complete BS. I really like the lady doth protest too much.

Christopher Penn 33:34
I mean, there’s your answer. I mean,

Katie Robbert 33:36
I’m also super skeptical of everything. So I’m an no one.

Christopher Penn 33:42
You’re not, though. I mean, it’s one of those things that if you have to say it, it’s probably not true. Whereas if somebody else says it about you, okay, like, I believe a lot more going on to a site like Glassdoor and seeing what it current employees have to say about the company rather than what the company has to say about itself. Like I Yeah, it’s like someone calling themselves a guru or a ninja like, Okay, you’re not either of those things. And I have the throwing stars to prove it. By but it is, I would be very cautious about about that. Because a realize that people understand what an NPS score is, as it relates to employment. B, it does make if you think about it from a non measurement person’s perspective, and so am I going to be like constantly monitored, if I work at this job, that they’re touting these weird scores, their job ads, like kind of tests where I have to take other bases to feed that data.

Katie Robbert 34:45
But I think depending on your role, you know, if you are like head of the customer support team, then it’s a relevant score. I used to work at a phone bank, I worked customer service. I’m sure all of us have at some point in our Our careers. And that was it was one of the things that I was measured on. Because as the person taking the call for the customer, you know, we were rated on, you know how quickly we got through the call the volume of calls, and if the person got the information and if it was accurate. And so that was sort of our version of the NPS scores. And so I can see where it is a relevant metric for certain job roles. But it does also feel like a really antiquated way of measuring job performance.

Christopher Penn 35:36
It, it can be, it can be useful to so there are one of the things that companies have been struggling with is scaling, right customer service is one of the hardest things to scale, because it requires it requires people and lots and lots of people. And if if you don’t have those lots and lots of people, and those people are not brained Well, yeah, it’s going to be you’re going to have some some trouble. This is where, by the way, and this is going to come full circle to a whole bunch of other things. There was a recent paper published by NIH, about generative AI and its use in customer experience. The the summary of the paper is that generative AI, on average delivers a better customer experience than the humans do. Because for a couple of reasons, one, there’s no time limit. Right? When, when you have a customer call center, one of my friends worked at a customer call center for a bank, and they had a five minute window, they they were penalized in their internal scores as an employee, if a call lasted more than five minutes, which to me is kind of nuts. So they go, Hey, someone broke into my account, you know, committing fraud going up, you have to solve in five minutes or less. And because of that, the reps also, were highly incentivized to end calls as quickly as possible, including just outright hanging up on people, which made the customer satisfaction scores go in the toilet. When you compare that to a generative AI system where you don’t have to worry about staffing, you don’t have to worry about five limits. And you can spin up as many instances of a chatbot as you want. Of course, the you’re trying to get better results because it’s on the customers pace and time then. And so you’ll see those NPS scores go way up, where the customer is effectively not on the clock, right, the customer is not being met, you know, penalized for having problems that require a lot of time resolve.

Katie Robbert 37:35
It’s, it’s interesting. Because customer experience and customer support is historically such a human thing. And I know our friends over at b squared, like that’s their whole platform is that it is a very human experience. And so it’s interesting that NIH came out with this study that you’re saying, Chris, that the generative AI is better than humans at the customer experience? Does it get into like what that specifically looks like? Is it answering basic questions? Is it you know, how, like, what is it that it’s better at

Christopher Penn 38:16
they were specifically measuring quality of responses. And this was both in marketing. And there’s a there’s another version of this paper that actually is in medicine, which itself is very interesting, where the ChatGPT responses were rated as higher quality, more informative and more empathetic, when ChatGPT gave the answer versus a human doctor, which was like, okay. But again, it’s a language model, it’s trained to deliver a language in a certain format. But that’s what we’re seeing with these tools. And so if, if you’re thinking about NPS scores and deploying them, one of the actions you might take is investigating to the possibility of, Can you can you divert, it’s called, literally called call deflection? Can you deflect or divert enough a certain percentage of calls to a machine that will solve those calls in a way that is satisfactory to the customer that makes them happier than the humans that leave that reserves the humans to handle the really difficult stuff? We’re like, Yeah, you know what, this is gonna be a 30 minute phone call between the Department company because you got a crazy fraud case to deal with. That’s where these NPS scores will help us measure different modalities of providing a good customer experience. And then also allows to re resource because now if I can divert 60% of your call volume away from your call center, the regressive company like great, we can cut 60% of our workforce, the progressive company will say, Okay, let’s now give our call center reps instead of five minutes per call us give him 15 minutes per call, and see if the call center NPS scores now go up because customers are going to get better treatment because the easy questions are being handled by machines.

Katie Robbert 39:59
What do you You say, John, are you? Would you rather interact with a machine? Or would you rather interact with a person when you’re calling customer support?

John Wall 40:08
Yeah, you know, it doesn’t matter, really, I just want like the answer as quickly and correctly as I can get it, which is the thing. But it’s very, it’s really interesting to me on the healthcare side of that, because there’s been a lot of research already and things done about how, you know, working in health care, fatigues the employees, you know, it’s like, if you have to spend your whole day dealing with burn victims, you know, eight hours of that, you just are not going to have the patience and the ability to do that. So, and we even see this at home with seniors to you know, things like Alexa, being able to Alexa will tell you what time it is 500 times a day, you know, and she doesn’t care, and you’re not going to lose her temper with whoever’s in the house. So yeah, there’s a lot of customer service stuff that should be outsourced like that. Because, yeah, infinite patience in the machine is actually something that’s going to improve experience for a lot of people. And then it’s, as Chris mentioned, like, you save the humans for the stuff that’s a complete dumpster fire that’s never been seen before. You know, that’s when a human can cut through it faster. But for the Yeah, the, the 650/4 time somebody has called you and you tell them to reboot your phone, you know, you really don’t need a human to be doing that. Exactly. We have to give Alex a shout out though, for his NBS, he said, is that a subset of what we should be scoring?

Katie Robbert 41:34
Yeah, that’s what I think of those scores. I think they’re NBS. So it’s, you know, we definitely we at Trust Insights have some modifications to make to our own data collection system. But I feel like I have a better grasp of why we do it in the first place. And I feel like that’s always, you know, where a company should start, like, it’s great to collect this information. But, you know, go through the five P process. What is the question you’re trying to answer by collecting this data in the first place? Because it might be different. So you know, it might be, you know, how efficient is our customer support? Versus how likely are you to recommend us to someone else? So those are two different questions. They might have the same structure to the survey, but you’re asking very different things. So you want to be sure that you’re clear about that, and that you have a plan to actually use the data.

Christopher Penn 42:27
And that’s, that is the number one takeaway, how we’ll plan to use the data. Data that you just collect is just a waste of time. So have a plan to use data. And, to your point, Katie, make sure that you know what that use is make sure you’re clear on the purpose of it so that you can use it intelligently. Yeah. All right. That is going to do it for customer satisfaction, scoring data, and stuff. We look forward to talk to you folks next week. Thanks for watching today. Be sure to subscribe to our show wherever you’re watching it. For more resources. And to learn more, check out the Trust Insights podcast at trust insights.ai/t AI podcast, and a weekly email newsletter at trust insights.ai/newsletter Got questions about what you saw in today’s episode. Join our free analytics for marketers slack group at trust insights.ai/analytics for marketers, see you next time.

---

Need help with your marketing AI and analytics? Get help with your AI strategy » Learn generative AI for marketers with our new course » Prove the ROI of your marketing with our advanced analytics services » Just got a "silly" question you want an answer to? Ask us here »	You might also enjoy: Empower Your Marketing With Private Social Media Communities The Intelligence Revolution: Large Language Models and the End of Marketing As You Knew It The Woefully Incomplete Book of Generative AI Instant Insights: The Beginner’s Generative AI Starter Kit How AI is Changing Marketing Keynote
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! Click here to subscribe now »	Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. Click here for Google Podcasts » Click here for Apple Podcasts » Link for all other podcasting software »