In this episode of In-Ear Insights, Katie and Chris discuss how to pick the right AI tools, answering the common question, “Which AI tool should I be using?” We explain starting with the problem, not the tool. Katie recommends the 5 P’s to evaluate AI readiness. We emphasize choosing AI intentionally, testing tools against requirements.
[podcastsponsor]
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
- Need help with your company’s data and analytics? Let us know!
- Join our free Slack group for marketers interested in analytics!
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher Penn 0:00
In this week’s In-Ear Insights, one of the most common questions that we have all the time heard this a bunch at the most recent marketing AI conference, MAICON in Cleveland is the perennial.
Which AI tool should I be using? Which I actually will should I use? There’s so many of them.
There’s hundreds of new ones every single week.
So Katie, when somebody says to you, hey, which AI tool should I be using? What do you say to them?
Katie Robbert 0:25
I say that is absolutely the wrong question to ask.
I counter with, what problem are you trying to solve? Period? Not? What problem are you trying to solve with AI? So, you know, we’ve talked about this a few times AI, at least in this context is like any other shiny object technology.
Now AI definitely has staying power in terms of how it’s going to be integrated into our culture, our society our work.
But putting that aside for a second, if you are leading with, what problem am I trying to solve with AI? You’re already going to come up with the wrong answer.
And so people who are asking what tool should I use? It’s not going to solve a problem, it’s not going to integrate into their process nicely, because they first need to understand what is the question they’re trying to answer.
And so Chris, unsurprisingly, I’m gonna throw the five P’s at them.
I’m gonna say, what is the purpose? What is the problem you’re trying to solve? Who are the people involved? What is your existing process look like? What platforms are you currently using to solve the problem? And what is your measure of performance? How do you know you were successful in answering the question? And then you can take a look at all of those pieces, and figure out where AI fits in.
And then from there, you can start to look at okay, AI fits into this particular spot right here, what tools are available? That answer that question, and then you can start to pick a tool.
So that would be the way that I would approach it.
But so you were at MAE con, a couple of weeks ago, Chris? I’m guessing you got this question on, like, what tools? Should I be practicing? On? What tools? Should I know what AI tools, you know, are the thing?
Christopher Penn 2:14
Yes, I got that question a lot.
And the answer, the answer is very similar to yours.
Right? It’s like saying, which appliances should I buy? Well, if you say if I say blenders and you start and you want to make steak, you’re gonna have a bad time.
Right? It’s, that’s, that’s not going to be super fun.
So it comes down to what I’m trying to cook.
Because there are many situations where AI is the wrong solution, particularly generative AI can’t do a lot of the things people think it cat, right, it’s not magic, it’s just literally a Word Prediction Engine.
So with that in mind, I do think, again, a lot like cooking, you should have a a foundational understanding of what the different underlying models are, what they’re capable of before you go buying tools vendors, because there’s a gazillion and a half different companies that have all decided to start up, you know, AI, this AI that, you know, they’re all 2995 A month or what have you.
I mean, 9% of those companies are not going to survive six months, because all they are is a UI on top of somebody else’s technology.
So it would behoove you to have a basic understanding of the foundational tools and technologies just to become comfortable with them and to say, Okay, this is what it is possible, this is what’s not so on large language models, the models I would suggest people get good at would be obviously the the GPT-4 series of models from OpenAI, which you can access through ChatGPT, I would suggest taking a look at Anthro picks Claude to model what you can get from claude.ai, which is good for really long form, data processing can handle 10s of 1000s words, for image generation, I would say start with something like Dali, to which you can actually get for free through Microsoft Bing, if you go to Bing image creator, you can use it up to 100 times a day, totally for free.
And that will give you a good foundation of image creation.
And those are the those tools I think are the good starters, to understand what the base technologies can do.
And then from there, once you figure out well, this task isn’t even a task that these tools can address or this task is at work, you just put there’s maybe some workflow things I would like to have integrated, then you can start shopping for vendors that provide extra features on top of the foundational models, but at the foundational model can’t even do it.
No vendor is going to be able to make you know make that appliance do something it’s not meant to do.
Katie Robbert 4:44
I like that perspective.
And I like that you want people to start with the foundation.
I wholeheartedly agree with that.
I sort of to to build on your cooking analogy.
It’s you know, like someone going to cooking school, you know, they’re not I’m going to start you off with, Okay, here’s a bunch of ingredients, good luck, they’re going to say you’re going to chop onions, and you’re going to chop onions for the next six weeks, until you can do it with both hands tied behind your back and your eyes closed.
It’s muscle memory, because you have to have those foundations, and then they’re going to say, you need to make a sauce using all of these onions, and then they’re gonna build on that before years from now or a month from now, depending on how quickly you learn.
They’re gonna let you sort of like do your own thing, but all of those foundational pieces, baking a standard loaf of bread, making a standard broth, making a gravy, you know, making an omelet.
Those are the things that you have to master first, before you can riff on it and sort of do your own version of it your own take, and those foundational pieces will never change.
The way that you make a standard omelet is always going to be the way that you make a standard on like the way that you make, you know, a bearnaise sauce is always the way that you make a bearnaise sauce.
And then you can start to you know, be creative on top of that, but the foundation will never change.
So I 100% agree with you, Chris, that if people are asking what AI tools should I be using? It’s not the skins or the UI that sits on top of the foundational tools.
It’s mastering those foundational pieces first.
And so you you mentioned Dolly for image generation.
You mentioned ChatGPT OpenAI for generative AI and what was the other one?
Christopher Penn 6:36
So ChatGPT and Claude, for for lack large language models, those two, because they’re, they’re very different services.
And they have different capabilities, it’s good to know both.
One thing I would say that is also really important in the AI space that it which is where the cookie analogy breaks down is that the fundamental models themselves change pretty rapidly.
For example, ChatGPT, rolls out a new version of its underlying models on a fairly regular basis every few months.
So imagine, you can’t like just keep getting a better and better frying pan.
When you work with a vendor of some kind.
That vendor may have technology that they’ve built on top of a model, but they’ve probably built it on top of, you know, a frozen pre trained model, because they need to be trained at for their purposes.
That can be problematic.
So again, at MAE con, one of the things that as we were walking around the expo floor, I noticed was some of the vendors, not all but some of the vendors are built on versions of GPT-3, that are now three years old, because they they build this whole infrastructure, and so they don’t have the agility to move to GPT-4 or to move to Claude two or any of the other models.
And again, so if you stick with the foundational models, when they change, you can change with them, you can learn what’s new, and what the enhanced capabilities are.
Whereas if you’re working with a vendor that’s, you know, maybe got a piece of software they built around the, you know, DaVinci three, which is one of the older OpenAI models, you’re stuck with that they’re stuck with that.
And so it will not be as capable as the foundation, the new foundational model itself.
So again, it’s another reason to stay close to the foundations, because they adapt so rapidly, you know that you don’t want to be using an older frying pan, if you’ve got a there’s a new frying pan that can cook even better and faster and more capably.
Katie Robbert 8:35
But I guess so there’s like, there’s, there’s, if we start to break that down a little bit, so there’s the, you know, the tool, the platform, the mechanism to create the thing, but then there’s the ingredients, the ingredients for the omelet, you know, your eggs, your butter, your salt, your pepper, so and so that hasn’t changed.
The speed in which the omelet cooks, or the temperature or you know, the, you know, the texture of the egg is going to change because of the frying pan because you have a newer pan versus older pan.
But you still have the exact same ingredient.
So I’m wondering if yes, the foundational models themselves will change.
But will the skill sets change? That’s the sort of question that says the ingredients and the models as the frying pan.
Christopher Penn 9:25
Yeah, so in this case, the ingredients are the prompts that you write.
So the prompt engineering and that those they don’t change nearly as much or as quickly.
In fact, if you want to get the guide that we have, we have one called Race framework that Katie you came up with.
If you go to trust insights.ai/prompts You can download the PDF totally for free now, not even a form fill up.
But though the structure of the prompts at least right now has been relatively stable for the last few years, with the understanding that In some cases, you may have to break up your prompts into different pieces to accommodate the different systems.
So for example, in ChatGPT, there are now these things called system prompts.
Or they may call them custom instructions, but the system prompts.
So that first part in in the, the Rails framework, your roll, that actually is something you can move into custom instructions if you want ChatGPT, to always behave like a certain individual, no matter how many other prompts you give it.
So there’s some nuance to it.
But those are not deal breakers.
And so those ingredients, like you said, are going to stay relatively consistent.
Now that said, how you use prompts and the evolution of your prompts that shouldn’t change, right? You’re just as as your your appliances should get better, your ingredients should get better over time as you become more successful.
So your prompts should be more successful over time, as you learn to refine them as you learn new tricks as you learn how to add words or take away words that can nudge a model into delivering what you want.
So if you’re saying, I want you to talk like the actress Florence Pugh, right, you might have to do some manipulation of like how you define the personality.
And that is that should always be evolving, because that is directly proportional to your skills with these things.
And again, one of the things that we strongly recommend is that you, you take time to, to keep a prompt library, somehow we keep one in one of our Google Sheets sites, it’s nothing super complicated.
But having those prompts there a is a good starting point.
And then B is an opportunity for people to collaborate and say, Oh, well, maybe I do this way too.
Katie Robbert 11:45
And I think that is a better answer to the question of what AI tool should I be using, then oh, well, you know, your neighbor down the street just created an app that you know, sits on top of Bing or dolly or something.
So you should probably use that one.
Because if you don’t know how to use the foundational tools in the first place, you may find an app or a piece of software that has a really easy, intuitive user interface.
But Chris, to your point, like there’s so many companies that are coming out with their own version, just sitting on top of the same databases that they’re not going to exist, or if you build your company or your services around one particular app that folds and goes under because it can’t keep up then now you have to go back and relearn those foundational pieces.
So I think the advice is learn the foundational pieces in the first place.
But what do you say to someone who’s like, but I don’t have time, or the skill set to learn the foundational pieces, I need something today?
Christopher Penn 12:54
I would say then you need to go back to the five P’s and ask what exactly it is you’re doing.
Because if you’re scrambling last minute to do something like that, then you have not thought this through.
And you may be about to buy something that you have not thought through that probably will not meet the requirements that that you actually needed to do.
Because again, a lot of these apps that are built on top of these foundational models, they have their own quirks, they have their own ways of doing things that may not align with your processes that may not align with the other platforms within your tech stack, that may not align with how your people do work.
And that evaluation process is really important to be able to say like, Okay, we’ve we’ve checked off here, the things that an AI based app must do, here are the nice to haves, right here are the things we don’t want.
And if you just need you’re going out and buying something, it’s like need you’re going out buying an appliance like oh, I bought this air fryer like Yeah, but we wanted you to make soup.
Like, technically, yesterday, airfryer can make soup.
But boy, are you going to be frustrated?
Katie Robbert 14:05
Yeah, that’s that’s a tough one.
I can’t even I can’t even justify that one.
That’s a tough one.
But yeah, it’s it’s basic due diligence, you know, you need a new refrigerator.
You should probably measure your doorframe.
You know, you need your food to be cold, you probably need a refrigerator, you sort of backtrack, the problem that you’re trying to solve, which again goes back to the five P’s, starting with purpose.
So if you’re asking the question or being asked the question, what AI tool should I be using? You really need to step back and say, What is the problem I’m trying to solve? And so if the problem you’re trying to solve is, well, we’re losing organic search traffic and so we think we just need to create a crap ton more content.
So what tool can help us create a lot more content so that we can get it out there and get our organic search traffic back, then, in that instance, the tool that’s going to help you create a lot more content, is it a generative AI tool like ChatGPT.
But it sounds like you’re not digging deep enough to understand what the real problem is, and if that’s going to be the proper solution, and so, so let’s, you know, just play out the scenario for a second, you know, you start with your user story of as a CMO, I want to create a crap ton more content, so that I can win back my organic search traffic.
Therefore, I’m going to use, you know, ChatGPT, the thing that you want to do in that instance, is make sure you’re going through each of these five P’s.
So the purpose problem I’m trying to solve is organic search, traffic is down, I’m gonna throw ChatGPT.
at it, people, I need someone who’s going to create prompts a prompt engineer in ChatGPT, to create a crap ton more content process, I’m going to look at all of the keywords I’m not ranking for, give them to my prompt engineer to put into ChatGPT.
And create a crap ton more content platform, it’s going to be ChatGPT, it’s going to be my website.
And performance, I need to measure whether or not I’m getting more organic search traffic back.
So if that’s the scenario, that you want to play out by choosing the tool, then you need to at least make sure you’ve covered those five P’s.
And that you have a measurement plan in place to know whether or not your choice was the right one.
Christopher Penn 16:34
Exactly.
And then when you’re done with the five D process, you make a list, these are the must haves above the platform, choosing, these are the nice to haves, and these are the Do Not wants, because there may be things that you might not want.
Or you might say like we can’t have something like that because of a legal or regulatory thing.
For example, if you’re working with customer data, you cannot use that with ChatGPT, you just can’t there is no way to to protect that data in a way that will satisfy regulations like HIPAA, for example, or fic regulations.
So that process is really, really important.
And that list of things must have nice to have cannot have or do not want, you got to be super clear on that before you go.
Before we go tool shopping, it’s like, you know, like the example you’re giving with a refrigerator like it cannot be wider than 36 inches, or it’s not going through this door.
Unless you want to knock down walls to your house.
That’s a that’s a cannot have.
Katie Robbert 17:33
Well, and what’s interesting to me, Chris, is that I feel like with AI being more accessible.
People have forgotten the basics.
And so, you know, when they choose a CRM, for example, a customer relationship management system, or they choose an accounting system, they typically go through this process of here’s what I need, here’s what I don’t need.
And so what’s what’s sort of baffling to me is why this particular type of technology is not being viewed in the same way of let me create my must have, and can’t have lists the same way that I would for choosing any other piece of software that goes in my tech stack that I’m going to be paying for.
Do you have thoughts about that?
Christopher Penn 18:18
Part of the reason for that is that AI is too big an umbrella term, right? Even generative AI is too big an umbrella term, you being specific, I want to use a large language model, or I want to use an image generator or a video generator or a sound generator.
Being more specific about that reduces some of the ambiguity because AI is such a broad blanket term that it almost people mentally just think it’s magic, right? It’s computer magic.
Well, no, it’s not.
It’s it’s mathematics.
It’s it’s statistics.
It’s probability.
That’s all these things are.
But that lack of understanding of the capabilities means it’s much harder to go through the five key process, right? If you don’t know what an appliance is capable of, you don’t really know whether it can is the right tool for the job or not.
Right? If you have this magical contraption, like what is it? How does it work? Right? You’re like, maybe it does this, maybe it does this, let’s let’s try it out.
And you find out that in fact, it is an air fryer, and that soup is not really something you want to be doing it.
And so the solution, a solution for that is, again, by taking some time with the foundational models, you’ll understand, oh, this is what a large language model is capable of hear the things that can’t do.
Right? It has no, by itself has no ability to connect to the internet.
So if you want to be doing something like summarization of today’s news articles, the a model by itself, it just can’t do it.
It needs infrastructure around it to do that.
So having having that fundamental understanding, then we’ll clarify for you.
Why, what things you need, and what things you can and can’t do with them.
Katie Robbert 20:04
I just I’m so I guess and I, it’s hard because I think Chris, you and I are so much closer to AI than a lot of other companies and a lot of other marketers that it feels like common sense to both of us to ask different questions to really start with, you know, what is it you’re trying to accomplish? Versus, Hey, what is this thing over here? Do Hey, what’s that tool do? Oh, that’s cool.
That one’s shiny.
And I think that, you know, we certainly go through that ourselves with in other contexts, but in this particular one, we’re both approaching it very pragmatically of okay, but what, what is the process currently look like? Where we would introduce AI, and it would have a positive or negative outcome on, you know, the overall service that we’re providing.
And I really want to encourage anyone listening to this podcast, anyone who is considering an AI tool to really dig deep, and ask those questions, do your requirements gathering? First of all? Why do I think I need artificial intelligence, and Chris, to your point, it might not be the right solution, you may need some basic automation, or maybe there’s a breakdown in communication between team members or stakeholders.
So if we go back to that example of as a CMO, I want to create a crap ton more content, so that I can regain back all the organic search traffic I’ve lost.
If you start to dig into the five P’s based on that user story, you may find that somewhere between people and process, there has been some kind of communication breakdown between the keywords that we’re losing traffic for, and the kind of content that we’re actually creating and that our customers want.
That’s not a problem that artificial intelligence is going to solve.
That’s a conversation between stakeholders and team members that has to be solved first, before you can introduce artificial intelligence on top, what you would actually be doing is exacerbating the problem by introducing artificial intelligence and creating more of the wrong content and then losing even more search traffic.
And by then you’ve probably let go of half of your writing staff.
And you’re sort of up a creek without a paddle.
Christopher Penn 22:26
Exactly.
And that’s the thing with a lot of these tools is is people make assumptions about what they’re capable of, because they don’t know.
And then when the tools don’t work out that way, they’re disappointed or angry or whatever, because it’s like getting angry at your blender that can’t make steak like that.
That’s not what it’s for.
Right? It’s, I mean, yes, you can technically cook the meat with a blender, but boy, are you going to be unhappy.
And that’s a fundamental misunderstanding of how the tool works, is it gets we kind of keep going back to you understand what foundation the foundational models are capable of, so that you know what to not use them for.
Katie Robbert 23:06
So let’s say I came to you and I said, Chris, what AI tools should I be using? And the answer is learn the foundational things.
Where does someone get started with that? Are there courses? Or are there trainings? Or are there talks or events or webinars that people can watch to get started? Because it can be overwhelming? If it’s brand new to you? So what is the what is your recommended starting place? What is a resource?
Christopher Penn 23:37
So there’s a few different ones.
I would say there’s a back the keynote talk that we gave at the MAE con Conference, which is on the Trust Insights website, you can go and grab that there.
It’s also up for now it’s at where can I get the slides.com That’s a 45 minute, just overview of the space in general, that’s a good landscape thing.
We have a completely free book, it’s called the woefully incomplete book of generative AI, which is also on the Trust Insights website.
Go to trust insights.ai/new AI book and that is a walkthrough of the sort of the different categories and with a lot of examples, a lot of ways to think about the stuff and then the third thing is start working with this stuff.
Just dig in you’re not gonna break anything you’re gonna blow stuff up.
If you have questions go to the go to our Slack group go to analytics for markers trust insights.ai/analytics For markers because I we’re talking about and stuff so much, they’re actually thinking we might even just want to start an AI specific channel.
We talked about it so much.
It’s not analytics that per se, but it’s something we’re thinking about for us internally, but that’s there’s there’s 30 to 300 people in there.
And obviously AI is a very hot topic in that space.
So there’s a lot of the a lot of different resources we put out.
There are Some other really good resources, the marketing AI Institute, we’re very good friends with with those folks as you go to Marketing AI institute.com.
They’ve got their intro to AI stuff as well.
And then again, get some hands on time, go download the prop sheet that we have, so that you can you have a structure to begin with, and start testing start writing.
Professor Ethan, Molotov said, treat these things like a very bright, very eager intern on their first day, they have no idea what you want them to do.
So you have to be clear and specific, but they’re very smart.
They’re very capable.
So you don’t have to dumb things down too much.
You just have to be very specific and what you want the intern to do.
That’s what allows as software packages out there, they’re very smart interns, read them as such.
Have conversations, ask them, Hey, can you do this? One thing that I think people forget is you can say, Hey, intern, and this ACO ChatGPT.
Schmuck, your work did you do as I told you to do? And very often they’ll go, oh, I apologize, I missed this part of the prompt.
And they will regenerate and correct themselves.
You can also ask them, Hey, how would I prompt you to do X tasks can actually have that conversation, they helped me out here, I don’t understand what I’m doing.
Again, like you were talking to the intern, they’re very smart into the very smart intern might be able to cook steaks, just so you can say, Hey, do you know how to cook a steak? And the intern would be like yes or no.
Katie Robbert 26:30
I would also plug if you’re really unsure where to start, you can bring myself and Chris into your organization for workshops and trainings and talks, you can reach us at trust insights.ai/contact.
Chris, you’ve been doing quite the roadshow and you have quite a few more stops ahead of you get doing this exact thing, educating companies on how to get started with artificial intelligence.
I’ll be doing a talk early next year, about how to assess the AI readiness of your team.
And so if you want to reach out to us, we are certainly happy to help as well.
Christopher Penn 27:07
Absolutely.
In fact, we just did a thing yesterday, if one of our customers and they they’ve got some really good ideas about how they want to be thinking about these technologies.
But again, we want to make sure that they’re doing it responsibly, we want to make sure that they’re doing it in a way that scales in a way that is safe.
And one of the things that is important to understand for folks, and this was in today’s newsletter, the Trust Insights newsletter, is these things can take on tasks.
And eventually that can erode some of some parts of your job.
So you want to think through that.
And understand the the people implications.
And the five fees.
I’m saying the people implications of these of these tools as they continue to evolve.
And again, that’s something that we can help with.
It’s something that we talked about all the time.
So there’s a lot of different resources that are available.
And then there’s also a lot that are less credible.
The easiest way to spot a less credible resource is any kind of too good to be true claim.
Right? When you hear us talk about AI Yes, it’s pretty cool.
But there’s like an asterisk, and it’s like a pharmaceutical commercial.
Niels MAE calls this this, this this hour can kill you.
Anyone who’s making claims about AI, and there aren’t disclaimers.
I would be cautious of like, Hey, you can 10x your business with generative AI today.
And yes, it makes seven figure income.
Okay.
No, you’re not a credible source.
Katie Robbert 28:43
I think that’s solid advice.
Well, thank you, Chris.
I think that this gives people a lot to think about and a lot of places to get started.
Christopher Penn 28:51
Exactly.
And like I said, if you’ve got comments or questions about all things AI and analytics, pop on over to our free slack group go to trust insights.ai/analytics for marketers, where you have over 3300 other marketers are asking and answering each other’s questions every single day and arguing about the Oxford comma and whether pineapple should be on pizza.
And wherever it is you watch or listen to the show.
If there’s a place you’d rather have it instead, we probably are there go to trust insights.ai/ti podcast where you can find us on most major channels.
Thanks for tuning in, and we’ll talk to you next time.
|
Need help with your marketing AI and analytics? |
You might also enjoy: |
|
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! |
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. |
Newsletter
Podcast
