So What? Solving data problems with Generative AI

So What? Marketing Analytics and Insights Live

airs every Thursday at 1 pm EST.

You can watch on YouTube Live. Be sure to subscribe and follow so you never miss an episode!

 

In this week’s episode of So What? we discuss how to use Generative AI to your benefit, the best tools for solving data problems, walking through Python 3 and ChatGPT. And if you don’t know how to code, we’ve got your back! Catch the replay here:

 

In this episode you’ll learn: 

• What generative AI tools are best for solving data problems
• How to do data analysis with generative AI & Python 3
• A walkthrough processing a spreadsheet with ChatGPT and Python 3
• What to do if you don’t know how to code

Upcoming Episodes:

• TBD

 

Have a question or topic you’d like to see us cover? Reach out here: https://www.trustinsights.ai/resources/so-what-the-marketing-analytics-and-insights-show/

AI-Generated Transcript:

Katie Robbert 0:29
Well, hey there, everyone. Welcome to so what the marketing analytics and insights live show I’m Katie joined by Chris and John, how are you fellows?

Christopher Penn 0:37
Hello!

Katie Robbert 0:39
A double wave today from John!

John Wall 0:41
Hurray.

Katie Robbert 0:45
Today we are talking about solving data problems with generative AI. We talk about this a lot, internally, because the questions that I always have for Chris usually result in it doesn’t do that. It doesn’t do that. That’s not how it works. And, John, I know you’ve run into this as well. You know, and so there’s a lot of questions that we get also from our audience around. What can I do with generative AI in terms of my data? Can it you know, clean up my Excel spreadsheet? Can it analyze my data? And so today, we’re gonna take a this episode to cover what you can do and how to do it. So Chris, where would you like to start with solving data problems?

Christopher Penn 1:27
Let’s start with a question you got at your most recent webinar that you just gave not too long ago. And it was this question that came in, he came in after the bell. So just as folks are hanging out this, this question came in, and so no one got a chance to tackle it. The question is exactly what screen it was, during a ref. The reference, I think you were talking about the different six different use case categories for generative AI, and you were recapping them. And this person asked, How about Excel spreadsheets? Do you have any suggestions as to how, for instance, humanize a list of URLs to display a summary or title aside the actual URL?

Katie Robbert 2:05
And so my question, what I would have asked for clarification is, what do you mean by humanize a list of URLs? I assume, you know, given their additional context is that it’s basically to extract information from the URL in terms of what is the title of the page? What is a quick summary of the page based on what you can glean from that URL? Is that what this person is asking?

Christopher Penn 2:38
I think so. Now, the one of the challenges to this that would would have helped, you know, make this a little bit more clear is there was no real we don’t really understand what this person was trying to do? And I’m sure you probably would have come up with it, at the very least ask them for a user story.

Katie Robbert 2:58
Oh, absolutely. I would have said, well, what is it? What is the question you’re trying to answer? What is the problem you’re trying to solve? Can you rephrase this question as a user story. And so one example of that could be as a marketing manager, I want to use generative AI to extract the webpage titles and summaries using a list of URLs. And that to me is like, Okay, I understand what you’re trying to do. The category of use case is extraction. Or it could be extraction and summarization.

Christopher Penn 3:41
Exactly. So this goes along two different lines of AI. So let’s, let’s dig into how you might solve this. So just just for for giggles Katie, how would you tackle this? If if you were left to your own devices? How would you go about doing this? Let’s say I have a list of URLs like this beautiful, lovely, huge list of URLs.

Katie Robbert 4:05
I like how this is the giggles portion of the show, because everybody knows that I’m going to get the answer wrong. So let’s all play along with today I’m going to play the role of person who is not an expert in generative AI. Let’s say that that’s what I’m doing. So what I would probably do, Chris is first I would start with opening an instance of generative AI probably either a ChatGPT or anthropic clog looks like we’ll use ChatGPT for today. And so I would put in my prompt, and I would say, you know, I mean, you’re gonna you construct them way faster than I do, but this is sort of on the fly like today I am playing the role of a marketing manager. My task is to or my action is that I need to take this list of URLs and find out what the web page names are in a quick summary of what the content is on the webpage. For context, this is going to be used to make, you know, decisions about who to hire. And then could you put the title, the webpage title and summary? In a downloadable Word doc, something along those lines, and so, you know, and then I would attach the file. So that is how I would approach it now, is this going to work? I have no idea. So but as we’re gonna find out, that’s the fun part.

John Wall 5:53
here we go.

Christopher Penn 5:55
But I think this is a really important thing to do. And the way you’re tackling it is exactly right, which is you’re asking how you even do this, right? This? Because it may turn out ChatGPT says, yes, absolutely, we can do this. Let’s go and, and do it. So of course, the first thing has got to do is import the spreadsheet will make sure that actually contains URLs, which is good, I suppose to proceed with turning the page titles and summaries for each URL. Now use a browser tool to visit tool, extract the information in the compiled to new document, this process can be done in batches to ensure manageability and access. So there’s the manual, so it’s gonna go out here and search, I started the processing, here’s this, the summary of the first three URLs from your list.

Katie Robbert 6:40
Wow, I am really impressed with myself gonna do a little pat on the back, that I didn’t totally blow up the system. But and I think that I think that one of the things, at least this is helpful to me is that it’s all just sort of, you know, a prompt engineering and using generative AI and asking for tasks is just a series of logic. And so making sure that you’re closing the loop on everything that you’re asking it and giving it proper context and information, which is why the race framework is so useful. And if you’re looking for a copy of that, you can get it at trust insights.ai/prompt sheet, where we give you the race framework and instructions on how to use that when you’re using generative AI.

Christopher Penn 7:28
Exactly. So now, let’s just quickly open up the first 10 URLs on that spreadsheet, which are five best answers to a public alternative topic research. What is social media publishing for Agorapulse Headless SEO explained enterprise SEO metrics? And how important is success? search engine ranking versus SEMrush? You will note that ChatGPT has completely hallucinated the answer so I was so close.

Christopher Penn 8:04
So your approach was correct. Asking the tool to do what we wanted to do was correct. However, it made it up. It’s completely just completely made this thing up.

John Wall 8:18
It was getting the URL so even. Uh huh.

Christopher Penn 8:21
That’s what it said.

Katie Robbert 8:23
You know, in some ways, I feel like, I feel like this is a really good example. Because my concern with the approach, you know, that I took, is that somebody would see that and go, Oh, that’s really easy. Let me do that. Copy, Paste. Great. Here you go goodbye. And because the alternative is, you know, and I’ve been in this in this role before is that you’re given a list of URLs and asked to manually copy and paste, you know, all this information from each individual page. And then, you know, in this instance, like, Oh, great works already done, boom, bang, no QA. And I think that that’s such a really important part of any thing you’re asking of generative AI is that you have to build in time to actually double check the work, you know, it’s like an unsupervised toddler. If you’re not double checking to make sure that they’re not sticking their fingers in the light socket, they’re totally going to do it.

Christopher Penn 9:22
Exactly. So we have run into what is a pretty severe limitation here, which is that it has hallucinated all of the URLs on this list. So for context, I didn’t get I didn’t tell where this came from. This comes from our content curation tool. So we have a piece of software we wrote in this last week’s content. And you can see these are all articles that are relevant and the kinds of things that we’d want to be sharing some enterprise SEO metrics and so on and so forth. So we’re really good list of, of content. However, ChatGPT could not actually process that list.

Katie Robbert 9:59
It looks Like it’s pulling all pages from our own website instead of the external sources. Correct.

Christopher Penn 10:06
All right, because So, okay, go ahead. What it did was it looked at the domain, and basically inferred from its own knowledge base. Oh, this is about Trust Insights. So here’s a bunch of Trust Insights content.

Katie Robbert 10:20
It’s probably going, please, please, please don’t check my work. Just think I’m really smart.

Christopher Penn 10:26
Exactly. And this, because the way our content curation system works, this list of URLs, these are all redirects. And these are all redirects that go to other places, but we have it going through our system. So we can track clicks on it to see which content gets the most interest, which we measure in Google Analytics. However, ChatGPT clearly could not handle that.

Katie Robbert 10:51
Okay, so in this instance, Katie, one ChatGPT, zero, because my approach was correct. And the system got it wrong. That said, the overall result is incorrect. So it really is still on me to fix it. So my, what I would probably due next is I would probably, you know, look at all of those URLs, pull the original URL, put those into a spreadsheet and try again.

Christopher Penn 11:24
Yep, you could do that. Having done this and try this, it will then hallucinate additional things, because because the nature of this task, and this is a really important clarification, even though we’re using a language model, and we’re dealing with words, this task is not a language task, this task is not generation, or summarization of language or extraction of language, this task is a is extraction of data, non language data. So is making HTTP calls from the HTTP protocol to another server two, and then getting waiting for the server response back. Right. So which is code, it is not language. And therefore, this task using just ChatGPT, by itself, or any of the models, is going to fail, almost every time.

Katie Robbert 12:21
So to go back to the question that that was being asked of, how can I take the URLs from an Excel spreadsheet? to humanize them? The answer is you can’t.

Christopher Penn 12:36
You can’t do it directly in the language model. That’s correct.

Katie Robbert 12:41
Interesting. Does that surprise you, John?

John Wall 12:44
Yeah, the other thing that I was thinking about is, and I think I’ve seen this, it make the same mistakes as if you gave it a more technical description. As far as go to these pages, grab the h1 tag, and grab the title tag and throw those in a spreadsheet. But this was what I was talking about this morning. This was one of the first things that I was doing with AI models was trying to get it to kind of go out and scrape and do stuff. And Chris was like, yeah, no thing you’re looking for.

Katie Robbert 13:11
And I think we’re all making that mistake, it’s still a really hard, please, for me, it’s still really hard to wrap my head around. You know, Chris, you talked about it. It’s a language task, what that means, you know, and how you really define those. So, alright, what can we do? How would we approach answering this question? Obviously, we’re missing some steps.

Christopher Penn 13:33
We’re missing some steps. And what we’re missing is we need to, we need to use the power of language models to generate language. But we need to generate a language that can address the domain of the task, which in this case, means writing code, because a service like ChatGPT, in addition to writing English and French and Swahili and Danish, can write code. And in fact, something like 70% of the usage of a tool like ChatGPT is to write code because it turns out that writing code is something it’s really good at. In most languages, but the language is the strongest in by a longshot is Python. So just as ChatGPT Can’t count, or do math, because those are math operations, not language operations. ChatGPT can’t go surf the web. But ChatGPT can write the code to surf the web.

Katie Robbert 14:29
Hmm. Okay. What does that look like? Oh, boy.

Christopher Penn 14:35
So let’s go ahead and start a new session here. Today, we’re going to write some Python code. The, the code will be in python three, which you should install on your machine if you don’t have it already, because you can’t run this inside ChatGPT And we’ll use it Popular Python libraries, like requests and beautiful soup. to crawl a list of URLs. From that list of URLs, we want to extract the HTML page title, and perform some extractive summarization of the top one to two sentences, from paragraph tags on the page, probably using some kind of natural language processing capability package, like Spacey. From there, we will take the URL, the page title, and the one to two sentence summary, and write that to a CSV file as the output. Now, this is a lot of very specific stuff. And there is some domain knowledge required here in terms of knowing what Python can and can’t do. If you don’t know that you can ask ChatGPT I want to do this what? How would I do this in Python? And we’ll walk you through lengthy explanations.

Katie Robbert 16:28
What if you didn’t know that it was even Python that you needed to do it? And could you say, here’s the task I need to accomplish? How do I do that? Would it would it give you would it start by saying well, first of all, you need to write some code in Python. And then you would say, Okay, can you do that? And it would say, Sure, why not?

Christopher Penn 16:50
Exactly, exactly. And and Chachi. If you don’t specify ChatGPT, whose native language is Python, if you don’t say, specify what programming language you want, you will default to Python.

Katie Robbert 17:02
Okay, go ahead. I think the interesting thing here is that, you know, we’re, we’re all starting with feeling like we have to know the exact thing to give it to do when really what you’re saying is, it’s okay to ask the system, what do I do?

Christopher Penn 17:19
But shot part of ChatGPT is the part people forget about crazy. So this ending sentence here is probably one of the most important things you can throw in any of your prompts ever, when it’s a task based thing is, what questions do you have, because if you don’t, it will try to execute based on what it thinks its best interpretation is, and it won’t actually think through the process. And we want it to think through the process.

Katie Robbert 17:50
So basically, it’s like a human. It kind of wants to get to the point.

Christopher Penn 17:56
Right? It’s, it’s behaving like a, you’re a really smart software development intern. And so part of part of working with it is this part this, which is the part that all of us in software development hate, which is requirements gathering.

Katie Robbert 18:15
All right, so let’s start seeing it has, you know, it’s listing out the components and asking clarifying questions, which, wow, how great is that? One, it looks like you still have to have some domain expertise. Because you know, the question, do you already have a list of URLs, I would have said, Yes. And then moved on to the next day. So access some great limits. Again, it feels like to complete a task like this, you definitely need to have technical expertise, even your original prompt about, you know, the construction of the page where exactly you wanted to pull from, you had to have some understanding of how web pages are constructed in order to give it the direction to say pull the data from here. That’s right. Now, John, would you have approached completing this task similar to me, or would you have gone the Chris Penn route?

John Wall 19:35
Yeah, no, it’s great. I sit right in the middle. Like, I would go more technical, but I wouldn’t actually get it to work, you know, like, I would have to go install Python and I’d be missing some libraries. And you know, like, there’s no way I would know, that spacey needs to be the medium English variant. You know, these are all like, I get the concepts of all of it. But yeah, it would take me like four extra hours then Chris is hammering through.

Katie Robbert 19:57
I like how you literally figuratively sit between myself and Chris. But I think that that’s really, you know, an important thing to point out is that you may have people on your team who have a bit more technical understanding, you’re just not utilizing those people. And so it’s a really good opportunity as you get deeper into using generative AI for tasks like this to really do an audit of your skill sets to see who knows what, you know, I think, John, you are definitely one of the more technical people that I’ve come across. It isn’t a data scientist or a technician, but you have a really good grasp of how technology works and how to approach it. And so it’s one of those underutilized skill sets on our team.

John Wall 20:47
I’m always like, I really need to practice my Python. Let’s actually do so much more if I could, if I was just better with Python and never make the time for it. Right?

Christopher Penn 20:59
So good news is these days, you don’t have to be you just have to know enough to be able to correct errors.

John Wall 21:04
This is why it keeps turning up the heat on me. I’m like, Okay, now I really need to.

Katie Robbert 21:09
So Chris, you went ahead and answered all of the outstanding questions, did you? I didn’t see the end of your prompt. Did you say Do you still have more questions? Or is this enough?

Christopher Penn 21:19
I just gave it the answers. Okay, ran off from that. Gotcha.

Katie Robbert 21:23
I suppose you could probably get into, you know, a very long loop. If you keep saying Do you have more questions?

Christopher Penn 21:31
Yes, you can. And you can, because it understands you’re trying to do software requirements gathering, you can go down some pretty deep rat holes, like and what about concurrency? What about your failover? And clustering is like no, no, I’m just trying to process a spreadsheet here, but just run once.

John Wall 21:48
Yeah, exactly.

Katie Robbert 21:50
But depending on the software, that’s actually not a bad thing. And that could be a really useful tool for business analysts and project managers who are struggling to get deeper requirements from their developers. Oh, absolutely.

Christopher Penn 22:05
It’s so it’s an a fantastic way to it’s really a fantastic way to do requirements gathering for anything. So people who are project managers, having a a project therapy session with ChatGPT is is incredible way to to get good feedback to Have you have you should ask it, what am I forgotten? What are the things that, you know, I should have asked you.

Katie Robbert 22:32
I definitely could have used this tool about a decade ago.

Christopher Penn 22:37
Okay, how are we doing here. So it has created a Python script outline. And it has, it is identified the Python libraries, it has written some functions to summarize text to extract info, and the main function that will execute it, and then it has this here. So let’s go ahead and take we’re gonna take this, we’re gonna copy that, go over to our coding environment, we’re gonna paste it in. So let’s read through it real quick. So summarize text, it’s going to use the NLP function from spacey extract info, it’s going to look for the paragraph tags, go look for the title tags. And it has some error handling. So if it can’t do it, we’ll just keep going. It has the input of the CSV file and an output file, which is good. It tells it to load the correct model in spacey, which is a language processing library. And then it tells you it says, Here’s the URL that we’re going to work with. Okay, so let’s give this let’s make sure that we have our URLs on our desktop, which we do. And let’s give this an actual try and see what happens. See if it completely blows up or if it actually works the first time around the box. guessing it’s gonna blow off.

Katie Robbert 24:00
Oh, the anticipation.

Christopher Penn 24:05
Awesome. So that’s the path to the CSV file. And they know that for sure that that file is there and on the desktop. So what do we do just give up? We didn’t know.

Katie Robbert 24:21
It would make for a very short episode. If we did.

Christopher Penn 24:23
it would. It would. So let’s do two things. Okay. First, can you write out our requirements in bullet point, format like this? So what we’re going to be doing here is we’re going to have it spit out the requirements and there’s a very good reason for this. Well, there’s two very good reasons one, we want to essentially have it refresh its memory as to what it is it’s supposed to be doing and why by having it restate its requirements, it brings all this text back to the front of its memory, to say, Okay, this is what you’re supposed to be doing. This, in itself is one of those handy little tricks to use to keep it on the rails. I’m going to copy this. And here’s Trick number two, I’m actually going to put the requirements into the code itself. And again, the reason for this is pretty simple. I want to ensure that every time I give it, its own code back to it, it includes a copy of the requirements so that as it’s making changes, it remembers what it’s supposed to be doing. It’s not it does, it does go off chasing its own tail, which can happen an awful lot. Okay, so let’s take, okay, here is the error I got, I’m gonna go to our command line prompt here, we’re just going to copy and paste this whole thing. And here is my code. We’re gonna go into our coding environment. I’m gonna grab all of our stuff, paste it in, please resolve the error. And so in doing so, again, this is why we want those requirements in there, because we want to remind it, this is what’s supposed to do and, and if it’s not clear, now’s the time to fix it. It says the Go ahead. It says, Here’s what is happening and why it’s happening.

Katie Robbert 26:37
So when we first started this task, you know, we gave it a list of URLs, and we said, complete this, what should be very simple task. And it hallucinated every single answer. How do we know it’s not doing the same thing? Now? I mean, you’re building in, you know, you’re going back and forth with it. But how do we know you’re not just getting hallucination after hallucination? And you’re never going to get to the answer. You don’t.

Christopher Penn 27:06
You don’t know that. However, because you’re forcing it to write code, which is a language task rather than tried to do the task itself, it is less likely it’s going to do that because there’s a difference between English or a human language and a programming language. And the difference is the programming languages are much less ambiguous. If something is wrong, as we saw here, it just doesn’t run it and it but and it returns errors, the errors come back in a standard format. And this tool because vacuumed up so much code across the entire internet is much more fluid at fixing coding errors, then it is talking to a human being and trying to say, Well, what did you try? What did you even mean? What does that mean?

John Wall 27:50
Yeah, the great thing is so like this has read all of stackexchange like developers immediately cry out enjoy knowing that this is all here.

Christopher Penn 28:01
Exactly. Well quite enjoy, and to some degree suffering as well

John Wall 28:05
Because you get this part of the mix.

Christopher Penn 28:09
And the thing is, like, we’re doing this, right, none of us are Professional Coders. And so the question is, well, what do you need Professional Coders for if as long as you have enough knowledge to operate ChatGPT? And ask it these questions? Katie, I see you’re making faces there.

Katie Robbert 28:27
I am making faces. You know, it? That is a heavy question that I know, you know, we could debate all day long, I do think you need subject matter experts in order to cut down on the back and forth, you know, that, you know, people who actually understand, you know, Python code. This may be more efficient at writing it, but it doesn’t mean that it’s more correct. And so if you have someone who doesn’t understand how to write code in Python, you know, let’s say you gave this task to me, Chris. I could be here for like four days, trying to get it to work. Whereas someone like you, or someone who actually understands how to code in Python, might take more like 40 minutes. And so I feel like the blanket statement, you don’t need code Professional Coders on your team. If you have this is a dangerous statement. That should come with heavy disclaimers. Like, yeah, I could do this. It’s going to take a heck of a lot of time. And it’s going to cost the company a lot more money than just hiring someone who knows what they’re doing.

Christopher Penn 29:49
Exactly. However, as you can see, if you do know what you’re doing this dramatically speeds up the process of writing this code because I’m not a particularly I’m a terrible Python coder. I’m better at rather than ever Python, and we’re 30 minutes into this, this live stream now. And we now have working code, it would have taken me 30 minutes just to have framed out this on my own because they’re constantly having to look up, okay, how do I do this or how to do this, and even in cases in our this stuff, a lot of manual labor of rewriting this stuff, where if I have a tool that instead of can generate the 98% of the code that’s needed, and I can tweak the last 2%, I’m much more efficient.

Katie Robbert 30:30
Oh, sure. But the difference between you and me is you know how to write code. Even if you call yourself, you know, a novice, or sort of, you know, not the best coder, you still know how to write code. I don’t know how to write code. I know enough to look for certain things. But if you hand me Python code and say, What’s wrong with this, I would probably throw up in my mouth a little bit and then say, I have no idea.

Christopher Penn 31:00
So our code ran. Yes. Let’s see what we got here. So there’s our URLs. Let’s see, here’s the page titles. Already, we’re in much better shape. Those are the actual pages, because we did a quick test. And let’s take a look at the summaries. And the summaries are. And they’re not great. They are using the spacey extract of summarization technique, but it’s it kind of sucks. But it did the job. It did it. This is like this is like the mythological Genie. Right? From old, old Middle Eastern tails. It does what you asked it to do. Even if what you asked to is a really bad idea, it will do it. So in this case, I gave it a focus of doing extractive summarization, which is where you try and find the most relevant pieces of text in the body of text. For what we’re doing, that is the wrong choice, we should be doing abstractive summarization, which would call an API like ChatGPT xAPI, say, read this entire page, and then you write a net new summary of the page. The reason I chose extractive was mainly for time, because abstract, it takes a bit longer. But in this case, it did what it was told. And it did a very good job of it.

Katie Robbert 32:34
Yep, halfway through that statement, you lost me. And I think that that, but I say that, and I don’t, you know, part of it is, you know, the self deprecation. But part of it is really to demonstrate that, you know, there’s a lot of marketers more like me, who are, you know, being asked to do tasks like this, which, on the surface looks really straightforward, like, read these URLs and tell me what the what these pages are about. But it’s so much more complex than that. And so this is where I can see a lot of teams are going to struggle of, you know, is generative AI really the answer? Or could I have just could I have already gotten halfway through this task myself doing it manually? And they do that? Because they don’t have the technical skill set to learn? How to make generative AI work for them when they’re trying to solve these data problems? Exactly.

Christopher Penn 33:28
And that goes back to really good requirements gathering, right? It goes back to what we call the five P’s because if this is a one time task, you’re never going to do it again. In the last 30 minutes, yeah, you probably could just browse those web pages and done it yourself. Right? If it’s a one time task, if it’s not a one time task, if it’s a you’re gonna be asked to this every month, every couple of weeks, every week. Yeah, you absolutely should, should be looking at okay, well, then maybe we do need to use generative AI for this. And to what Gwen just said in the comments there, knowing that if this type of abstraction is something that you know, that your y coder should be involved, right, just being able to make icing for cake, you know, ingredients for cake, doesn’t mean you can make the cake. It’s absolutely true. This is true.

Katie Robbert 34:20
No, and so I wholeheartedly agree, you know, and Gwen is really just reiterating. You know, the point is that you know, there there is a learning curve with generative AI and so depending on the task you’re trying to do, first you need to do your basic requirements. And then you need to figure out, do I need a subject matter expertise? And so in this case, yes, I would need a subject matter expert and I would, you know, Chris, look to you, or someone with your similar skill set and say, you know, this is the thing I’m trying to do. You know, can you help me put this together in generative AI so that I can do it on a repeated basis? This because the information and the skill sets that I have today aren’t enough in order to complete that task using generative AI.

Christopher Penn 35:11
Exactly. One of the things that we say about generative AI are the six major use cases generation extraction, summarization, rewriting classification. And question answering, which, by the way, if you want to learn more about this, you can take our course on the topic. There’s a big asterisk that isn’t on this slide that which is what we just talked about the last 35 minutes, it has to be a language task has to be language tasks that you’re using these tools for, if you’re doing task, you’re trying to have it do tasks that are not language tasks. None of these categories matter. So yes, we are technically doing a form of extraction, extracting data from an HTML page. But the process of doing so is not a language task, because it’s not interpreting the language of HTML, and trying to pull it out. It is a that is a code based task. And that is not what generative AI does. And that’s why we had to write code to be able to do this. And once we are able to write the code, which again, we got this done in under 30 minutes, then it was able to fulfill the task as we asked him to do.

Katie Robbert 36:15
I’m gonna throw this out there. And this is maybe something you and I can debate on a podcast. But I did, you know, a very quick and dirty internet search. And the internet considers math to be a language. And I think that’s why people like me get confused all the time. When you say it’s a language task. I’m like, But math is a language. So what do you mean by a language task? And you know, this is definitely you know, we could go down a rabbit hole with this. But I think that’s where I personally get confused of, what do you mean, generative AI can’t do this. I have a data problem I need to solve. And it’s going to involve Math and Math is a language.

Christopher Penn 36:56
Yes, mathematics, particularly equations are a language. But computation is not a language. And so yeah, we can definitely talk about that on a future podcast episode, because it is an interesting distinction. It’s one of the reasons why, for example, gendered AI, thus far, has deeply struggled very, very unsuccessfully, to do things like compose music, music is a language, there’s absolutely musical language and structure, generative AI, because the underlying nature of the mathematics that powers it, can’t really do that. Well, it can do it. But it’s not great. I, for my my personal newsletter, last week, I had to create a pop melody, and I handed it to an actual musician. She was like, this sucks, like, dude, this really sucks. A seventh grader would do a better job than then then your machine did. She’s like, will someone you know, could could someone put a song out with this? Yes. Would it do well, on the charts? Maybe maybe not. But she says like the musical equivalent dentist drill, no one wants to listen to this. And again, that’s because there are there are complexities in the construction of music, that are not language, that are psychology, that are neurology, that are physics, that are not reflected in the language that appears on a sheet of music, just like there’s complexities in mathematics that are not reflected in a series of equations. Whereas those complexities don’t necessarily exist, to the same level, in the written word, or in a programming language.

Katie Robbert 38:35
So if we go back to the original question about using Excel, because where we started the episode of How do I solve my data problems using generative AI? I think a lot of marketers are trying to do things like this. Would we have been better off? Asking ChatGPT or whatever generative AI tool is? Here’s my question, can you do this? Or how what is the best way to approach it? That may have saved us some time?

Christopher Penn 39:07
You absolutely can. You absolutely can do that. And that wouldn’t be a bad place to start. Is to say like, what, what approach would you use to solve this problem? Because again, use the chat portion of ChatGPT.

Katie Robbert 39:22
What do you think, John? Are you going to start chatting with ChatGPT? More?

John Wall 39:25
Yeah, I gotta have it. Turn on my buggy code and give me let me run afoul of what else I can break.

Christopher Penn 39:35
Yep. Gwen said, You segways chat with ChatGPT

Katie Robbert 39:42
Yeah, yep. And, you know, I, I forget to do that as well. I, I feel like systems like gender like ChatGPT and anthropic Claude give a lot of us less technical folks, so much anxiety that we need that we kind of feel like we have to have the answers going in. So we forget about the chat component of when it says What can I help you with today? It literally means What can I help you with today? It didn’t say, what exact directions do you have for me today? It’s really trying to be a helpful system and guide you through. So I have some, you know, ideas I’m gonna play around with, with ChatGPT and even with Katy GPT, and sort of stretch and see where, you know, her knowledge, you know, extends to at this point.

Christopher Penn 40:31
Exactly. And as these models continue to advance, they will get better at anticipating needs. OpenAI has said in press briefings that they expect to release GPT five this year. Regardless of you know, all the hype and speculation, what we do know is it’ll probably have fresher knowledge, you’ll have better conversational abilities have better understanding of language, and, and more capabilities for things like vision, etc. But fundamentally, what it’s really good at is that conversational AI, so you will need you will be less and less, really tricky, prompting strategies to get good results, it may take you longer and be less efficient than someone who knows, you know, the underlying structure of the model and you know, how it navigates Leighton space their their pumps may be more efficient, and faster, but you will both get to the same destination.

Katie Robbert 41:27
I do not understand latent space and all the vortexes and, you know, whatnot that you just talked about. So I will be the least efficient probably on the team for a while, as I’m trying to figure out but I am going to challenge myself to use the chat function more. So, you know, as all of us collectively or you know, we’re looking at ChatGPT, or generative AI in general to help us solve our data problems. It sounds like really the first place to start is ask the system. Can you do this? Or how do you do this?

Christopher Penn 42:01
Yep. How would you do this?

Katie Robbert 42:04
And Gwen has one final question that I’m going to punt to you. I may have missed it. But is this the same case for co pilot? So Chris, can you first set the stage with what is copilot? And then?

Christopher Penn 42:19
Well, okay, so this is slightly tricky, because there’s a bunch of different tools that are named copilot, there’s GitHub copilot, there is Microsoft copilot, there’s Bing copilot, and stuff copilot is really Microsoft’s main brand. All the copilot services that Microsoft operates use the the GPT-4 model underlying them. So if you go to Microsoft, Bing, and use that the gender of AI there, you’re using GPT-4 through Bing, if you’re using an office co pilot in Microsoft Office that is using Microsoft Knowledge Graph combined with GPT-4. So because it has access to the same model, you can you can ask it the same question. So if you’re using Microsoft Excel or Microsoft Word, yes, underneath it is GPT-4. And so you can perform, you can ask the same questions like, Hey, I have this Excel spreadsheet. What would What’s your recommendation for how to convert this into a 10? Slide PowerPoint presentation? And what’s your recommendation for analyzing it to understand the trends in it, and it will tell you like, Hey, here’s some things that you can think of and try. Be sure to give it time to run. So one of the weird quirks of language models, at least today’s transformers based language models is that the longer and more verbose they can be, the better the results are. Because there’s a very technical explanation, but it basically boils down to, it needs time to think it needs it needs room to foam at the mouth until it dials into the correct answer. And the more you can do, the better. So if you ask more open ended questions on it. It will tend to perform better if you ask it, show your work step by step or let’s talk through this step by step or what do you think about this? Give me your analysis before you give me the answer. Right. These are all strategies for getting it to talk more, and as a result to dial into the answer faster.

Katie Robbert 44:14
You know what’s funny, Chris the same tactic works on people.

Christopher Penn 44:17
I know. People are just squishy versions of ChatGPT.

Katie Robbert 44:25
John, any final thoughts?

John Wall 44:27
I can’t beat that one today. I gotta win winner.

Christopher Penn 44:34
All right, everyone. Thanks for tuning in, and we will talk to you all next time. Thanks for watching today. Be sure to subscribe to our show wherever you’re watching it. For more resources. And to learn more. Check out the Trust Insights podcast at trust insights.ai/t AI podcast and a weekly email newsletter at trust insights.ai/newsletter Got questions about what you saw on today’s episode. Join us Free analytics for marketers slack group at trust insights.ai/analytics for marketers See you next time

Transcribed by https://otter.ai

---

Need help with your marketing AI and analytics? Get help with your AI strategy » Learn generative AI for marketers with our new course » Prove the ROI of your marketing with our advanced analytics services » Just got a "silly" question you want an answer to? Ask us here »	You might also enjoy: In-Ear Insights: The Trust Insights Podcast AI and Copyright Law: How Copyright Applies to AI-Generated Content The Woefully Incomplete Book of Generative AI How To Build Better Analytics Dashboards and Reporting Empower Your Marketing With Private Social Media Communities
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! Click here to subscribe now »	Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. Click here for Google Podcasts » Click here for Apple Podcasts » Link for all other podcasting software »