In this week’s In-Ear Insights, Katie and Chris discuss shiny object syndrome, blind spots in your marketing technology (especially around AI and machine learning) and how arrogance can lead to substantial technical problems in your tech stack and company culture. How can you avoid pitfalls and blind spots? How do you manage AI and machine learning initiatives, especially if you’re a non-technical manager? This and much more – tune in now!
[podcastsponsor]
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
- Need help with your company’s data and analytics? Let us know!
- Join our free Slack group for marketers interested in analytics!
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher Penn 0:02
In this week’s in ear insights, let’s talk about machine learning.
And AI.
talk a bit about this, and the array of different tools and techniques available to us and why I feel like some folks are doing it wrong.
I was talking with a technologist not too long ago, who was bragging about his company is so cool, they got this new model and state of the art and all this stuff, and put it through its paces.
And it wasn’t the results were substandard, the results were no better than something you could pick up off the shelf, and do yourself.
And in looking at the process, behind the scenes, there are a whole bunch of things, important prerequisites that were missing.
Specifically, they hadn’t done any of the basic pre processing, some filtering, some cleaning of the data, some sampling of the data to make sure the data was okay.
And as a result, the model that they created spit out garbage.
And when went to talk to these folks, there was this almost arrogance to their to their tone, like oh, no, we don’t need to do those things.
The software smart after we’ve got the state of the art software, it’s smart enough to handle that.
And they said, but clearly it’s not.
It’s still spitting out garbage.
And they got me thinking about the corporate cultures.
And that sort of tech arrogance, it’s not just a guy think it’s very much a tech thing around AI and machine learning and how we can avoid this and how we can prevent this because clearly, this company has spent millions of dollars and thousands of hours, put it together state of the art software that is not state of the art.
And that seems like an A, I’m sure their investors are probably not thrilled about this outcome.
So Katie, we know that there are a bunch of technical solutions.
But how do we fix that people culture of saying Well, I’m clearly you’re the smartest person in the room, I know what I say goes, I must be right.
And then seeing results, which are clearly not the best.
Katie Robbert 2:11
you slap them.
End of story and a podcast.
Now you know it, what you bring up is a really challenging issue because you’re trying to change people changing tech is probably one of the easier things to do.
Changing people is one of the more difficult things and more often than not the issues that we see within an organization really stemmed from that people problem.
And so you know, you’re describing a very typical scenario, the I know best, you can’t change my mind.
And there is no immediate solution to that.
It is something that at least in my experience takes time to start to change people’s opinion of themselves and the work that they’re doing.
And a lot of it comes down to, can you present back to them real life scenarios where this thing is not working? Can you present back to the the data that shows this is the way that you approached it, this was the result.
And this is how it’s working.
Versus had we done it this way, some alternatives, and this would have happened this way instead, or I did some mock up and projections had we done requirements gathering due diligence, ahead of time, this is the money that we would have saved, you know, so there’s different ways to approach it.
And it’s really understanding your audience.
So if you’re the person tasked with trying to change the opinion of the person who is high and mighty on themselves and think that they can never be wrong, first of all, I feel bad for you.
Because that is a really difficult thing to do.
But second, it’s it’s more than just saying No, you’re wrong.
You need to get over yourself.
It’s really using psychology and reading the person and understanding how they respond.
And so it’s it’s not an easy issue to tackle.
Chris,
Christopher Penn 4:07
how do you handle that that tech bro personality, because in this particular instance, there was a entire area of domain expertise they’re completely lacking, that would have made their lives easier, it would have saved them a lot of money and gotten them better results out of the gate.
But the person in question was was so blinded to that.
So they could No, no, no, we don’t need to understand the Oh, this particular area of of machine learning.
We are software’s better than what we created is better than that.
And it is that outright dismissal of the Yeah, this there’s some older stuff.
But guess what, that older stuff actually works as good or better.
There are conclusive proofs in academic research that, you know, this machine learning model here offers maybe a half a percentage point better performance, then the The old reliable stuff that he has is a little clunkier to work with and is not sexy, you’re not going to win any awards, you’re not going to get any press for it.
But it works.
So how do you? How do you handle that tech was like, No, no, we don’t, we don’t look at the old stuff.
We’re all we’re all in on the new stuff.
Katie Robbert 5:18
The first mistake that people make is trying to prove someone wrong.
So if you’re dealing with that sort of aggressive personality of I know everything, you can’t tell me otherwise, meeting them where they are, and saying No, you’re wrong, is just going to put them on the defensive, and likely, they’re just going to shut down and say, I don’t have time to listen to you.
The stuff that I’m doing is fantastic.
And so it’s really more about you know, and none of this is meant to sound, you know, devious or backhanded or malicious.
But it’s really starting to, you know, work with them and collaborate with them, you know, okay, so help me understand why this is the best solution helped me understand why this was the one picked, and you didn’t explore other options, because I need to understand and so it’s really working with them to open up and explain and in some ways, justify the decisions that they’ve made.
Because you need to also believe in the thing that they’re doing, because you need to help them, sell it or market it or, you know, do customer service on it, or whatever it is.
And so showing them that you are part of the team and that you’re on their side.
But you need more information to understand it is a really good, almost gentle way to get someone to open up and really start to explain why they made those choices.
What you’re trying to do, in some ways, is to like, disarm them a little bit, because again, in my experience that like arrogant tech bro thing is very much a facade.
And it’s almost like that armor of, you know, I’m not super confident what I’m doing.
But if I present myself as super confident, then nobody’s going to question it.
And they may not even be aware that that’s what’s happening.
And so starting to poke holes and attack them and say, you’re wrong, and you’re doing it wrong is only going to make the situation worse.
You have to approach it.
As we’re a team, we’re in this together help me understand why we’re doing it this way.
Okay.
Chris is now like, Oh, so that’s what you’ve been doing all these years?
Christopher Penn 7:26
Well, how does how does the whole shiny object syndrome thing to because that’s the other angle is not only there’s this belief in being correct, but there’s also this shiny object syndrome of saying, you know, this, the new thing must be better.
And in this case, the new thing isn’t better.
And it’s harder and more expensive than the old thing.
Katie Robbert 7:53
So, you know, I think that’s one of the reasons why having an AI, my dogs have, you know, Happy Monday.
I think one of the things that’s really important on a team is making sure that nobody is working in isolation.
And so that’s why having like, you know, what, I think they call it like peer development, and you know, a review committee, and all of those things are really what help you get out of that, you know, bad habit of the shiny object syndrome, shiny objects are great, new and innovative, things are great.
But you need somebody on the team who’s willing to question, Is this the right move for us? Is this the thing that we need to focus on? Right now? What problem? does it solve that we are currently experiencing? Or do we see a solution, and we’ll come up with the problem later.
And I think that the shiny object syndrome, quite often leads to, here’s a really cool solution, we don’t have a problem for it.
But it’s a really cool solution, if we can just use it, and then we can convince people that they need it.
And it’s the absolute backwards way to innovate.
And I think that, you know, again, it’s making sure that nobody is working in isolation.
So if you, you know, if you’re the CEO, and you don’t have an advisor, or a mentor, or someone that you can bounce ideas off of, then you probably need that person.
Because if you’re relying on the people, you know, who report up to you, they may not feel comfortable saying this is the wrong decision.
You know, and then if you are on the engineering team, the development team, the IT team, there should be people within your organization who can say, I don’t know that that’s the right thing, or can you help me understand why that is the right thing?
Christopher Penn 9:38
What do you do in a situation where you don’t have those size teams? Like for example, for us? There’s there really is and this has been the theme for a good chunk of my career is there’s no one to actually QA the code, right? Because there literally is nobody else who understands within our organization, what exactly is going on behind the scenes? was the same way that the previous place that we both worked where there was, I literally could have said anything and and people would have had to accept it because there was no one to look at and go, that that doesn’t look right.
Yeah, that’s it.
The outputs are have been good.
The results have been good.
So it’s not, there’s not obviously a flaming dumpster fire behind the scenes.
But at the same time, especially if you’re somebody who is managing a person like me, and you don’t have the technical experience to look under the under the hood and go, huh, it’s all those roaches doing? How do you manage? How do you manage that? Because there may be cases where that the person who’s writing that code may have a blind spot may have that arrogance may have that shiny object syndrome?
Katie Robbert 10:43
Well, and again, it comes down to feeling confident to ask questions, and they’re not antagonistic questions, they’re just questions of, you know, so that we can be on the same page and the same team around the thing that you’re creating, help me understand it and explain it to me in such a way that I can then, you know, talk about it on your behalf.
And you don’t need to be in the room because I want to be your advocate.
So I need to understand it wholly, I don’t have to be able to push the buttons.
But I need to understand the methodology, I need to understand and answer questions of why was this the methodology chosen and not something else? And this is a conversation, Chris, you and I quite have a lot so that I can do that in your behalf.
And so, you know, if in the scenario where you’re a smaller team, and you don’t have that, then that’s really where you need to rely on those communities that do exist.
And so maybe you belong to an engineering slack community, or, you know, I can’t think of the name of the other ones, the, you know, there’s GitHub, and then there’s
Christopher Penn 11:48
stackoverflow.com, overflow,
Katie Robbert 11:49
exactly.
Those kinds of communities are where you could actually do some of that peer review, if you don’t have that built into your organization.
Now, that works only if you are willing to take the feedback, if you are, you know, a little bit hard headed and say no, my stuff is great, then you probably aren’t open to doing that.
But then you also have to, you know, live with whatever the result is.
And I think one of the things that strikes me, as you were sort of describing this, Chris is, you know, I think some of the arrogance is well, the results look great.
So why would I do it a different way? Well, how do you know the results are the true results? Or are they just, you know, bias, because they’re the results that you were hoping to get.
And I think that there’s a lot to unpack in that.
Christopher Penn 12:37
There.
There really is.
And that’s a case, you know, the classical machine learning example, where somebody built, built this image recognition software to differentiate between a dog and a wolf, and it did great in the lab, failed miserably, the moment went into production.
And then once the post mortem was done, and they looked inside the model, the model was looking to see if there was snow in the photo.
And if there was snowing the photo is a wolf.
And like, okay, so the results in the lab look great, but not so much we see this a ton in medicine, you know, we see it in clinical trials, it looks great in the lab.
And then once it goes to trials, it goes that that didn’t perform as expected.
The risk there of bias is extremely high.
Because, again, we all everyone in this situation, whether you’re in a clinical lab, whether you’re doing machine learning whether you’re just doing your your marketing reporting, you have a built in incentive to have results that look good.
Right? That’s that’s just the nature of everything that we do I have an incentive to create software that works.
I can’t think of any organizations where there’s an incentive to to fail.
And when you’re encouraged to to fail without eventually succeeding.
And so the challenge then is, how do you tame that so that it’s still honest API like yes, this thing does fail more often than just like, we have this one client and the software that we built for this client.
The first five additions didn’t have error checking built into it.
The sixth edition actually has error checking built into it now and works much better.
But it took a while to get there.
And then I think that happened, because there was no process to build that in place.
Katie Robbert 14:23
Well, and that’s a really good starting off point.
And so this is something you know, that we talk about quite a bit is process isn’t meant to get in the way of innovation process is there as the foundation for innovation.
So you know, Chris, in your example, you’re saying that, you know, five versions of the software didn’t have error checking.
And then we finally said, Oh, that’s what needs to go into it.
So we updated our documentation and any new software moving forward has error checking built into it.
Therefore, when you are presented with a new data extraction challenge, you can say these are the steps that I I need to follow for the basic foundation to know that it will work.
And then I can innovate on top of it.
You know, and so I think that process is one thing.
And then the other is, you know, failure is such a taboo world, taboo word in the business world that, you know, it’s, it’s okay to fail.
And it’s actually better that you fail.
A couple of times, like before it goes into production, I think that people are, so want their stuff to go to production so quickly, that they don’t do that QA.
They don’t do that error checking.
And they’re just like, No, I’m confident it’ll work it has to work, it just has to work.
If I just like, put all my hopes and dreams in it, it’s definitely going to have to work.
And then it fails and then right, and then you kind of look like a fool.
But if you would spend a little bit of like r&d time and this is something that companies need to make the decision to set aside for, then you could get all those failures out of the way, and then build something really great that would go into production.
Christopher Penn 15:59
So to wrap up, whether you’re dealing with shiny object syndrome, hardheadedness, or fragile tech bro egos, at the end of the day, it’s the people that present the biggest challenges to your outcomes.
The technology is easy, and process is straightforward, even though there may not be easy, it’s managing the people.
So if you feel like you’re struggling to manage the people, go ask for some help.
If you haven’t already done so join our free slack group over here at TrustInsights.ai dot AI slash analytics for markers.
You got questions not only about analytics, and a tech and the numbers, but hey, how would you manage this situation? Or, hey, I could use some advice on on this and you don’t have that peer group.
come by and join us.
We’re happy to have you there.
If you have not already done so please subscribe to the show.
Go to Trust insights.ai slash gi podcast, subscribe to on Apple podcast, Google podcasts, Amazon, music, Spotify, Stitcher, you name it.
It’s over there.
Thanks for watching, folks.
We’ll talk to you soon.
Take care
|
Need help with your marketing AI and analytics? |
You might also enjoy: |
|
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! |
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. |
Newsletter
Podcast
