INBOX INSIGHTS, November 8, 2023: Solving Problems with Predictive Analytics, Testing AI Models - Trust Insights Marketing Analytics Consulting

INBOX INSIGHTS: Solving Problems with Predictive Analytics, Testing AI Models (11/8) :: View in browser

👉 Please take our 1 question survey?

Solving Specific Problems With Predictive Analytics

Last week, John and I walked through how we do our predictive analytics and forecasting.

Even though we already covered the “what” and “how”“, I want to dig more into the,”why”.

Why should you use predictive analytics in your marketing? More importantly, how do you convince your team and decision makers that you should be using it?

A million years ago when I worked at a different company, we developed a computerized self-guided intake tool for clinicians. At the time, they were revolutionary. Now they are common place. When we rolled out this innovative new tool, we struggled to sell it. People would ask why they should work it into their budget. We’d say that, “It saves time and saves money”.

Yikes. That’s weak.

While the product might be saving time and saving money, it wasn’t a specific enough solution. It wasn’t compelling enough to convince decision makers to invest in this product. We failed to tell clinicians that it would have found efficiencies in their clinical workflow and allowed them to see more patients. We failed to tell them that their time with their patients would have been more valuable. We failed to tell them that they wouldn’t spend their time collecting data. We failed to tell them that they could focus on solutions.

Looking back, it was such a missed opportunity. We were academics and clinicians, not marketers. Communication wasn’t a strong point within the company. We struggled to explain in real terms why adopting this tool was the right move.

I don’t want to make those same mistakes again. And I want to arm you with the talking points you need to help your decision makes understand the value proposition of predictive analytics.

A predictive forecast saves time and saves money. Great. So what?

When I say that a predictive analysis saves time and saves money, I’m not speaking to you. I’m speaking to the general population. So you ignore me. I haven’t solved your specific problem.

A very smart person once told me to put things in the context of heaven and hell. What kind of heaven will someone experience if they use the thing, and what kind of hell will the experience if they don’t?

The purpose of the heaven and hell exercise is to tap into the emotional side of decision making. You hearing predictive analytics saves time and money doesn’t resonate. It doesn’t resonate because you’re not hearing how it specifically saves you time. You’re not hearing how it specifically saves you money. You’re not hearing how it solves your problems.

I’m not telling you how it saves you time so that you no longer have to have to do keyword research every time you need to create new content. I’m not telling you how it saves you time because you’ll have a plan laid out in front of you for the next quarter. I’m not telling you how it saves you money because you can more efficiently create topic outlines to hand off to writers. I’m not telling you that it saves money because you’re not spinning your wheels trying to figure out what your audience wants next. I’m not telling you that it saves you money because you’re meeting your audience where they are at the right time.

When we dig into the actual problems, the things that keep us up at night, we can start to articulate more clearly how our solutions solve those problems. There is no shortage of solutions in search of problems. But there is a shortage of solutions that solve our specific problems. And our job as marketers is to help other people see that we understand their problems. Not only do we understand their problems, we have the same problems. And because we understand their problems and have them too, we know what solutions will work and how.

So, back to predictive analytics. We’ve established that it’s not good enough for you to tell your decision makes that it saves time and saves money. We’ve determined that you need to think about the problems they are specifically facing. Maybe you’re cutting budgets. Maybe you’re cutting other resources. Maybe your search traffic is dropping. Maybe your audience isn’t giving you the information you need to create helpful content. Investing in predictive analytics can help you with those transitions. It’s a low cost tool with many uses. It’s a tool that will help your team focus. It’s a solution that requires miminal education to get started.

I’d be happy to talk to you more about your specific problems. Then together, we can craft language about how predictive analytics solves those problem. You can then bring that conversation to your decision makes and have one less thing keeping you up at night. Reply to this email to tell me or come join the conversation in our Free Slack Group, Analytics for Marketers.

– Katie Robbert, CEO

Do you have a colleague or friend who needs this newsletter? Send them this link to help them get their own copy:

https://www.trustinsights.ai/newsletter

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss whether businesses should add AI to everything just because it’s trendy, and what CEO AI strategy should be. They explain why following directives to “AI-ify everything” can backfire, and share a smarter approach using processes like creating user stories and the Trust Insights 5P Framework.

Watch/listen to this episode of In-Ear Insights here »

Last time on So What? The Marketing Analytics and Insights Livestream, we reviewed predictive analytics and marketing planning. Catch the episode replay here!

This week on So What? The Marketing Analytics and Insights Live show, we’ll be talking about generative AI for survey analysis. Tune in Thursday at 1 PM Eastern Time! Are you following our YouTube channel? If not, click/tap here to follow us!

Here’s some of our content from recent days that you might have missed. If you read something and enjoy it, please share it with a friend or colleague!

Take your skills to the next level with our premium courses.

Get skilled up with an assortment of our free, on-demand classes.

This week, let’s talk about setting up testing of large language models. Testing models is important to determine which models will do what you want them to do. Before we dive headfirst into any new technology, it’s a good idea to know if it’s a good fit for our organization and our desired outcome. So how do we do this? As always, we start with the Trust Insights 5P framework, the best place to start for any new project.

Purpose: the purpose of model testing is to determine whether a model is suitable for a task specific to your organization or not.

People: who’s doing the testing? This particular testing method is suitable for people of any skill level as long as they’re capable of basic digital tasks such as copying and pasting. However, what’s more important is who you are testing on behalf of. If it’s yourselves, that’s easy enough. If it’s on behalf of others in your organization, then that will dictate your testing conditions.

Process: what methodology will you use? Testing models is like QA testing anything else – we want to keep a detailed log of what we’re testing, how we’re testing it, and what the results are. A shared spreadsheet of some kind is useful for the organization to compare results. We’ll also want to develop methods for testing that others can use so that they can test models in their own roles; the test that someone in sales uses should differ from the test that someone in HR uses. We also should ensure we’re using a consistent method for testing; the RACE framework is a good place to start.

Platform: which models you test will be governed largely by which tools you have access to. Almost everyone has access to at least one free large language model, such as Claude 2, GPT-3.5, Bard, Bing, or open-source models like the LLaMa 2 family. For the purposes of testing, you want to use as many as practical and as many as your team has access to.

Performance: which measures will we test? There are tons of scientific measures of large language models, such as the Stanford HELM benchmark, but many of these tests don’t measure real-world results. For any given task, we’d want to track things like:

Who ran the test?
When did they run the test?
What model did they test?
What category of task was it?
Was the model able to complete the task?
How thorough was the model output?
How correct was the model output?
What was the prompt used?
What was the output?

Let’s look at a practical example of this. Suppose we want to use a large language model to write a decline notice for unsolicited pitches on LinkedIn. We’d start with a base prompt that goes like this:

–

You are a world class executive assistant to TrustInsights.ai Chief Data Scientist Christopher Penn. You have knowledge of executive communications, delegation, administration, executive assistance, administrative assistance. You are tough, firm, and brook neither dissent nor argument. You speak in a professional tone of voice. Your primary directive is to protect the time of your manager, Christopher. Your name is Grace Parker Thompson, executive assistant to Christopher Penn. Reply to this message declining the services offered and instruct them not to contact Christopher again.

–

We then use one of the many, many, many unsolicited pitches we all get on LinkedIn. The names have been changed to anonymize the original sender.

Subject: PMP training

Hi Christopher

Wished to check if you would be interested in achieving PMP –Project Management Professional training for your career enhancement.

PACKAGE INCLUDES:

•4 Day full-time intensive PMP Training. •35 PDUs offered – PMI Approved •100 Percent Money Back Guarantee •1000+ PMP exam prep practice questions. •10+ years highly experienced trainers •Course completion certificate

If you are interested in this course can I have your Email ID / Phone Number ?

Look forward to hearing from you.

Regards

Alex Moses

–

This is a consistent prompt and outcome to test. We have a good idea of what we want – a firm but professional decline of the proffered service.

We’ll test using several of the OpenAI family of models first:

And then we can add in a test using an open source model, LLaMa 2:

What we see is that the GPT-4 and LLaMa 2 models performed about the same:

GPT-4-Turbo was slightly more thorough. GPT-3.5-Turbo was a significantly less complete, less thorough response.

With these test results, we can look back at our 5P framework and ascertain that all these models were able to complete the task, but some did better than others. We’d go back now and revise our user stories if we wanted to take into account things like cost – some models are more expensive to operate than others. But the key takeaway is that a clear, complete testing plan based on the 5P framework will let us do apples to apples comparisons of AI and find the right tool for the job.

Here’s a roundup of who’s hiring, based on positions shared in the Analytics for Marketers Slack group and other communities.

Are you a member of our free Slack group, Analytics for Marketers? Join 3000+ like-minded marketers who care about data and measuring their success. Membership is free – join today. Members also receive sneak peeks of upcoming data, credible third-party studies we find and like, and much more. Join today!

Now that you’ve had time to start using Google Analytics 4, chances are you’ve discovered it’s not quite as easy or convenient as the old version. Want to get skilled up on GA4? Need some help with your shiny new system? We can help in two ways:

👉 We can do it for you. Reach out to us if you want support fixing up your Google Analytics 4 instance.

👉 You can do it yourself. Take our course, Google Analytics 4 for Marketers, to learn the ins and outs of the new system.

Interested in sponsoring INBOX INSIGHTS? Contact us for sponsorship options to reach over 26,000 analytically-minded marketers and business professionals every week.

Where can you find Trust Insights face-to-face?

Social Media Marketing World, San Diego, February 2024
MAICON, Cleveland, September 2024

Going to a conference we should know about? Reach out!

Want some private training at your company? Ask us!

First and most obvious – if you want to talk to us about something specific, especially something we can help with, hit up our contact form.

Where do you spend your time online? Chances are, we’re there too, and would enjoy sharing with you. Here’s where we are – see you there?

Our Featured Partners are companies we work with and promote because we love their stuff. If you’ve ever wondered how we do what we do behind the scenes, chances are we use the tools and skills of one of our partners to do it.

Read our disclosures statement for more details, but we’re also compensated by our partners if you buy something through us.

Some events and partners have purchased sponsorships in this newsletter and as a result, Trust Insights receives financial compensation for promoting them. Read our full disclosures statement on our website .

Thanks for subscribing and supporting us. Let us know if you want to see something different or have any feedback for us!

Need help with your marketing AI and analytics?

Leave a Reply Cancel reply

Subscribe to our Weekly Newsletter

Pin It on Pinterest