00:00 – Introduction 02:15 – The billable hour dilemma 06:40 – How artificial tools change the workflow 12:30 – Finding value in the process 18:15 – Call to action

Watch this episode to see how you can adapt your daily routine and charge fairly for your growing skills.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Katie Robbert: In this week’s In-Ear Insights, you get just me, Katie. Chris is on the road. And so we’re going to try something a little bit different. You get me, and as always with me is my chief marketing officer, Georgia. So she may or may not pop in.

This week I want to talk about billable hours. This is a hot topic in the world of consulting. It always has been. Billable hours is one of those things that people have always struggled to really define outside of legal, because that’s really where the precedent comes from, isn’t it?

When I worked at the agency, we were tasked with all of our work being billable. But when I looked at what we were doing, and this isn’t a knock at agencies, this is just the way that they work, we’re looking at this client has signed up for this much retainer. And this much retainer covers this much work. And so when you start to break down budgets at a very high level, you have however many people are going to be assigned to that particular retainer.

And then you have to calculate out, well, what is the bill rate for this for any individual? And then sort of back into the numbers. When I was working as a project manager, this is how we would do our budgeting. And so we would have software developers, we would have creative folks, we would have our marketing team. And every single one of those people has to have some sort of value assigned to them.

And that value then translates into how many hours they can contribute to any given project or task. And so if the task was we have to create a one-page brochure, that’s likely going to involve our marketing team and our creative team. Therefore, our software development team doesn’t need hours unless it’s something being put up on the website. So then they need to have time. And that’s like roughly how you would calculate the budget.

Now, when you’re talking about billable hours, you’re kind of working in the opposite. So you have this idea of how many hours any given person. So let’s say my bill rate was 100 dollars an hour just for easy math. And I had 10 hours to complete something. So of course I’m saying easy math, and I can’t calculate 10 times 100. So that’s a thousand.

So let’s say I have a thousand dollars. Let’s say my bill rate is 100 dollars an hour. So I have 10 hours that I can work with. What ends up happening when you have those billable hours is you say, okay, I have 10 hours. So I have to find 10 hours worth of work to bill, even if the work is only going to take me five hours. Because then I have this five-hour bucket left that I’ve already told them I’m looking at bill for.

Or if you’re just doing sort of time and expenses, you’re saying, I’m only going to bill you 500 dollars, not a thousand, like we originally agreed. Anyway, all to say that the billable hour is a really hard thing, especially now with AI. So AI has made us more productive. This is something that I’ve talked about in the newsletter, on the podcast, on the live streams.

It’s made us more productive. It’s allowed us to produce more. But now, where is the trade-off in that? So one of the things that I was reading this morning was Upwork, Future of Work Index 2026, how AI is redefining the value of work and skilled freelancing accelerates. So a freelancer is someone who definitely is doing hourly billing. So it’s going to take me X number of hours to do it.

I’m going to bill you for all the hours that I work. In their world, it almost doesn’t matter if they’re using AI or not. It’s however long it takes to get the work done. And if they’re using AI to get the work done faster, that’s what agencies are trying to go after. They want to get more work done faster so they can spend less money. The downside of that is the people actually doing the work and the people who are billing for the work are making less money. This is the rub, this is the hard part.

So in this Upwork news release from July 14th, so it is a little about two months old. New research shows that share of skilled knowledge workers who freelance jumped from 28% to 38% in one year as AI raises the premium on judgment-driven work and pressures execution tasks. There’s a lot to unpack there. So let’s go ahead and start to do that.

So if you think about a lot of the discourse that’s happening right now in the B2B industry or really kind of any industry, there’s a lot of pushback on AI generated content. So AI created anything. You have a whole camp of people who are saying no, thank you. They don’t want to see AI generated images, they don’t want any AI generated content. And we have a whole course on that helps you identify how people are using AI to write and how to flag this is written by AI.

So you can go to trustinsights.ai slash AI for writers to learn more about that. But back to the point, a lot of people, rightly so, don’t want to see AI generated content. And so part of the reason for that is because all of these large language models have been training on publicly available data. A lot of that publicly available data is still owned by an individual artist or an individual creator, but it’s being trained on in any way.

So you’re running into a lot of copyright infringement. You’re running into a lot of work that is being passed off as original, but really it’s AI. A really good example of this is, and I apologize, I don’t have the article up, but there is a story about the two mathematicians recently who’ve been working for years. Their life’s work was to solve this particular equation.

And they were very close to solving it, but a large language model swooped in, solved it for them, and then took the credit. So their whole life’s work has been reduced to nothing because the large language model said, hey, I can do that. And this is where the friction is coming from. There are still people who want to be creating and owning all of the work solely without the assistance of AI. The flip side of that is that companies are not willing to pay now for the amount of time that that manual handcrafted work is taking. So where does that leave us with today’s topic of billable hours?

Well, I think again, I think back to when I was working at an agency and our time was in billable hours. It was really tricky to think through what was I doing for an entire hour. So depending on the person, you may have someone who’s really efficient, you may have someone who works a little bit slower. You’re never going to get sort of that equal playing field of an hour is an hour is an hour.

And so, if you spend an hour on a call with a client, that’s one billable hour. But then you take another 10 minutes to clean up the conference room, bring your stuff back to your desk, answer a couple of emails, and type up the notes from the meeting. So that’s where does that 10 minutes go? It gets really hard and we start to estimate. Well, if you get to 10 minutes, then you estimate up to the 30-minute mark.

If it’s anything less than that, then it doesn’t count. That only gets you so far because then who’s eating that time? That goes into the overhead of the organization that you work for. They’re then eating that time and they’re not going to be happy about that. So if you think about that billable hour and what you’re actually doing, if you’re getting that work done now with AI and it’s getting done in a fraction of the time, can you bill that whole hour? It’s a really good argument for why value-based pricing is more effective, but it’s also harder to say, well, why is it priced this way?

So value-based pricing in a nutshell is essentially you’re paying for the expertise, you’re paying for the output, regardless of how the methodology was to create the final output, which it has its own set of issues. So, for example, if you’re paying for my expertise and Chris’s expertise, you have combined over 20-plus years of expertise in AI alone. Then you bring in analytics, then you bring in organizational behavior, then you bring in management, you have all of those different pieces that is unique to just myself and Chris. So regardless of how long the work takes, that’s what you’re paying for.

So we can be super efficient, or we can take weeks and weeks and weeks to create something. You’re not paying anything different for that. So there’s a definite benefit to value-based pricing only if you can prove that the person who is pricing it is actually the expert. And that’s a whole tricky thing. So back to this notion of how is AI changing the billable hour? Because it is changing it.

So what used to take maybe 30 minutes after a meeting to draft up notes is now done instantaneously because we have all of these AI transcription services. And so the notes just already exist and they’re cleaned up and they’re summarized and they’re ready to be sent out to everybody else. And it can be done automatically without you even sort of interjecting as that human in the loop to say, let me now read it and send it. They can send automatically. So how is that something that you can bill to a client?

One of the things that you know we just mentioned was what does it look like for legal? Legal being sort of the precedent for where billable hours started. Guess what? Legal’s changing. What’s driving legal AI pricing in 2026? This is an article by Cleo. So legal AI tools range from free to more than 1,200 dollars per seat per month. These are the things that are replacing the basics of law practice. So you have solo practitioners, who are trying to buy these tools.

The things that it’s replacing is sort of the basic research. It’s the basic pulling together of information, past precedents, past cases, it’s summarizing things. That’s the kind of work that maybe a paralegal would do, maybe a junior associate would do. And now law firms are turning to AI to replace those things. Is it still billable? Well, that’s what’s up for debate. Everything we’re coming to is that AI is changing the way that we’re thinking about everything we’re doing in terms of work.

So I don’t want to just sit here and pontificate, but basically, the thing that I come to, the issue that I have with billable work in general, with or without AI, is that a lot of it is subjective. So it’s a really good opportunity to look at your process for everything. And of course, I would be remiss if I didn’t bring up something like the 5P framework by Trust Insights. So you have purpose, people, process, platform, and performance.

Purpose, what are we doing? What’s the question we’re trying to answer? People who’s involved, process, how are we doing this? Platform, what tools are we using, and performance. Did we do the thing? When we think about what needs to be billable, this is a great place to start because not everyone is going to be on the same page. So let’s say you have a weekly report due to a client, and you know, this week it takes you one hour, next week it takes you two hours.

The week after it’s going to take you 30 minutes. You can’t just average it out and say, I’m just going to bill 90 minutes to the client, because that’s on average how long it takes. If they start to really scrutinize, that’s not going to work very well. So this is an opportunity to bring in the 5P framework by Trust Insights and really make sure that your process is clear. You can even start to put in timestamps to say this should take about this much time. This should take about this much time.

It’s like a recipe for baking or for making a dish. You know, we love our cooking now cooking analogies. If you think about gathering the data should take no more than 20 minutes. If it’s taking more than 20 minutes, that’s when things are going wrong, or there’s some sort of an anomaly, or there’s a technical issue. Those are the things that you want to factor in. Then assembling the report takes 30 minutes, then analyzing the report and then writing up the final pieces. That’s you know, all of your different time blocks.

Bringing that into AI, if you have an API, now the data collection is instantaneous. It no longer takes you 20 minutes, it takes you maybe 30 seconds because the machine has brought it all in. Assembling the report, copying and pasting slides might have taken you 30 minutes. Now it takes the machine about five minutes of processing, and you didn’t have to touch anything. You were in the kitchen making coffee, pulling the insights. I would highly recommend you still do this. But if you’re having the machine do it, something that may have taken you another 20 minutes to sort of go through and pull out all of the insights could take the machine. Again, it’s going to take less than a couple of minutes.

And then writing up the after action and the, you know, the next steps that might take you, the human, another 10 minutes to say, well, what do we want to do about it? It’s going to take the machine split seconds. So what was typically an hour now takes a machine maybe, maybe 10 minutes start to finish to do. So how do you bill for that? Well, this is where you want to make sure you’re keeping the human in the loop. So instead of just handing everything over to the machine, the client who is paying for your time and your expertise, maybe you’re offloading the part of the data collection.

Maybe you’re offloading the part of assembling the report, but you’re not offloading the part of the insights and the actions. What can happen is now you can spend more time really thinking through. Well, what do I know about this client? What do I know about this industry? Getting deeper into the insights of not just here’s the surface level of what I’m looking at, but what else can I bring into this that’s going to make it more useful? And then making sure that those actions are really and truly something not only that can be done, but that has an impact against the business goals and that are measurable.

That’s the trade-off when you start to think about well, it has to be a billable hour. It doesn’t take me an hour anymore. What is it that I can do for an hour? This is the conversation we’ve been having for a long time, a couple of years, about, you know, will AI take my job? Well, it’s going to take parts of it. It can take parts of the tasks, it can take the data collection, it can take the report assembly. But you, the human, should be retaining the critical thinking. I’ve been looking at AI as an opportunity to really get deeper into the thinking that I just didn’t have time to do before.

So I’ve been able to really focus on where are we going with the company? What are we doing well? What should we be doing less of? But the thing that was taking up a lot of my time was actually pulling together all of that data, pulling together all of that information. I can have the machines do that for me now. And now I can focus solely on what do we do about it? What does it say? A project that I was working on last week was cleaning up the data in our CRM. So our customer relationship management system and understanding how are people finding us? Because we have about eight years worth of data.

And I haven’t been able to confidently answer the question of here’s the channel that works, or here’s the thing that is a true buying signal. Using a large language model, I use Claude specifically, I was able to start to clean up and pattern match and say, this is the data that needs to be cleaned up, resolve that. These are the pieces that fit together. Let’s put that puzzle together. And now here’s everything. That by hand would have taken me weeks to do. Instead, using a large language model, it probably took me a couple of hours. So if you think about that billable time, you’re going from weeks to hours.

However, the trade-off is now I have this rich, clean data set that I can spend weeks if you’re sort of going by that sort of like one-to-one replacement building actionable campaigns against, doing deeper research on. I can now say this is a buying signal. If this is a buying signal, how do I get more of that? And spending my time there versus spending all of my time gathering the data. And that’s a bit of the trade-off. So again, when we get back to this notion of billable hours, yes, AI is changing the game, but you need to adapt with it. You need to be thinking through how does this look different? So if you’re still stuck in the, well, I bill for an hour because I sat in a meeting, that’s okay, but you can do better. And I encourage you to do better.

If it’s I bill for an hour because the report takes an hour, but now it only takes 10 minutes and I don’t know what to do, you’re really not taking advantage of the time savings that you’ve gotten from the AI system. This sort of brings into that other topic of how do I measure whether AI is working? So the billable hours of billable hour. You only have so many hours in a day, and you only have so many hours you can build, it’s what you’re doing with that time. So again, if the report used to take an hour, now it takes 10 minutes, great, that’s time savings. However, if nothing is happening with the report, if you’re not taking action on it, if you’re not learning more, then all AI did was save you time in assembling it. You didn’t take advantage of that extra 50 minutes that you got back to actually do something with it and move closer towards your business goals.

So that’s the way I want you to think about those billable hours. So, yes, AI is changing the way we’re thinking about billable hours, but there’s a really big opportunity for humans, for the critical thinkers, for the experts to dig in deeper to what is all of this information mean? So, if you’re not taking advantage of it, I highly recommend you do. So if you have thoughts on the billable hour, I definitely want to hear about it. What did I get right? What did I get wrong? What other information should we be thinking about? You can bring some of your comments into our free Slack community, trustinsights.ai slash analytics for marketers. Free to join. We’re active just about every day. There’s always a question of the day. It’s fun. If you want to learn more, you can go to our podcasts where this one is hosted, trustinsights.ai slash TI podcast. You can catch us on YouTube at TrustInsights.ai slash YouTube. If you want me to never do a solo podcast ever again, let me know. Or you know what? Maybe don’t. Maybe I just won’t do it again. Either way, we’ll catch you next time.

Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI.

Trust Insight services span the gamut from developing comprehensive data strategies and conducting deep dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology, and Martech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, Dolly, Midjourney, Stable Diffusion, and Metallama.

Trust Insights provides fractional team members such as CMO or data scientists to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights podcast, the Inbox Insights newsletter, the So What Live Stream, webinars, and keynote speaking. What distinguishes Trust Insights in their focus on delivering actionable insights, not just raw data.

Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations, data storytelling. This commitment to clarity and accessibility extends to Trust Insights educational resources, which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI.

Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 03:15 – The trap of picking platforms first 08:42 – Building a self-updating AI memory system 14:30 – Replacing heavy tools with simple scripts 21:05 – Balancing cost and environmental impact 27:50 – Call to action

Press play now to uncover the practical strategies that will transform how you handle technology.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In-Ear Insights, let’s talk about local AI and the reasons and, well, the implementation of it. We’ve talked in the past, and you’ve seen in the Trust Insights newsletter and on LinkedIn and all the places we post, about how local AI is one of the ways that you can reduce the environmental impact of generative AI by doing things on computers under your control, either on your literal laptop or maybe specialized desktop devices like the Nvidia DGX Spark. But either way, you don’t need a massive hyperscaler data center that vacuums up entire rivers and electrical power grids just to write email summaries. Katie, you’ve been on this journey for a bit now. What have you found and where are things going for you?

Katie Robbert: It’s been interesting so far. One of the goals that I had when I started this journey was to document everything so I could share it in the Inbox Insights newsletter, which you can subscribe to at TrustInsights.ai/newsletter. Each week I plan on going through basically the big things that I’ve learned, which to be honest, is a lot. One of the first things that became very clear to me is that I was breaking my own rules when it came to the 5P framework by Trust Insights. And so if you’re not aware, you can go to trustinsights.ai/5p-framework. The 5Ps are purpose, people, process, platform, and performance.

And so in this journey to understand what it looks like to migrate to local AI, I was leading with the platform. I was doing what we highly recommend everybody not do. And I was doing it. And I want to be very clear, and I want to call myself out because it’s such an easy trap to fall into. We’re like, oh, I should move to local AI because it’s going to be greener, it’s going to save the environment, it’s going to do this or that. I already chose the platform. I already chose the solution before really exploring the other P’s. And so, acknowledging that—I didn’t realize it until I was about partway through creating the first set of requirements. When I looked at it and said, I started with the platform. What am I doing? This is ridiculous. So all to say, this is why we talk about the 5P framework the way that we do.

So where I started was with a user story. As a persona, I want to say that a user story is a simple three-part sentence: As the CEO and an environmentally conscious citizen, I want to figure out where I am negatively impacting the environment with my use of AI without losing productivity, so that I can feel good about the work that I’m doing. Kind of convoluted in my writing, it’s a little bit more clean, but that’s the gist of what I was trying to get. I don’t want to lose productivity, and I want to do better for bigger than my own little ecosystem. What I found out is it’s not an all-or-nothing. It’s not just migrate everything to local AI and you’ll be fine. I had to really dig in.

So the first thing I did, aside from setting up the projects and the folders and all that good organizational stuff, was Chris. I borrowed your scaffolding from the Data Diaries, which is in the Inbox Insights newsletter, and you talked about the product requirements document. You talked about the technical specifications and you talked about the work plan, and you gave prompts for those. So as a user, I borrowed those and adapted what I needed to because I wasn’t coding; I was creating something different. And one of the things that you did in that prompt that was really helpful in the product requirements document was you gave the instruction to the AI that says, interview me until you have enough information. That’s where I’ve been spending most of my time.

Because there are about twenty questions. The questions that came out were ones that I overconfidently thought I knew the answers to. And then when I really stepped back to think about it, they weren’t the case at all. For example, moving to local AI is going to be more environmentally sustainable. That might be the case. However, based on the nature of the work that I do in my process, that’s not the best move for me. And it’s something that I need to really consider. And one of the things that came up, and this is where I want to get your thoughts, Chris, is I rely really heavily on the memory that you can create within a project, specifically in something like Claude Desktop Co:Work. And then you have the projects, and you can actually commit information to memory to reference later.

This is not true of every functionality of every iteration of every large language model. But this is something that I specifically use because one of the things that I’m trying to do is remember back in January, we had this great idea that we said we were going to do and then it died. And now it’s August, almost September. What are we going to do about it? I’m able now to go through lots of different files from different systems. Basically one of the good, solid use cases of generative AI is summarization and categorization of the data. So it’s not making decisions for me, but it’s taking everything from Slack, from transcriptions, from emails, from wherever—all these different places—and saying, I’ve summarized everything; here’s all the stuff. And I’m like, that’s amazing.

Let’s commit that to memory so that tomorrow when I’m like, hey, let’s do this thing over here, it’s like, remember yesterday? Because I don’t remember yesterday, but the machine does. So that for me is such a big use case. And where I got blocked on my journey to local AI is how often the memory on the local version gets updated with things that are happening near real time. I want to kind of get your thoughts on that, so I’ll just stop there. That’s one of the learnings that I’ve had so far.

Christopher S. Penn: Memory is writing things down, and that’s essentially one of the core functions that a lot of people don’t do with AI. Then they figure out, oh, these tools have no memory, and you constantly reinvent the wheel. And so there are a bunch of technical solutions for that when you’re using local AI that you can create. In fact, you can create a better memory with local AI than you can with the cloud-based ones because the cloud-based ones have pretty strict limitations in terms of how many resources you’re allowed to consume when you’re on your laptop. You can do whatever you want. There are some things you should not do, but the system will let you do them. So that’s one of those sort of bookmarks that you’d want to mentally say, okay, well what are the different options?

In fact, Google just had a paper on this from DeepMind three days ago. Essentially instead of having things loaded in a project, you have your local tool create—in whatever folder you’re working—have it write a wiki with links to markdown files within itself. As you make changes to the project, it updates the wiki because it can traverse it and say, okay, I’m on the homepage of the wiki. And now I’m in Katie’s projects. And now I’m in Katie’s dog projects. And now I’m in Katie’s fence, the backyard project. It can keep just drilling down as though it were a person browsing a wiki until it gets to, oh, on August 26th, Katie decided we’re not going to fence the backyard with an electric fence. We’re going to use an old-fashioned vinyl fence for that. And the advantage of that kind of system is that it’s very human-readable so that you can browse your own wiki and go, I said that.

Katie Robbert: And that’s what I’m doing with the cloud-based version right now. So the question for me as I’m making these decisions is what does my process of maintenance look like to make sure that the local version is staying as up to date as possible, since things are happening every single day that I’m trying to remember? I’m one person. And one of the things that I’ve been sort of thinking about is, I’m one person, one laptop, one project. What does this look like at an enterprise level when they’re trying to do something similar without getting too far into the weeds of that? Right now, a big consideration I have is, yes, I can create that on a local environment. How often am I keeping it up to date, and is that taking more time and more resources to do that than it is to do it in the cloud? And again, these are questions that you probably know the answers to, but I, as the person who’s trying to learn, don’t yet know what that looks like.

And so for me, that’s been an incredibly helpful exercise because here’s the thing, I have a Chris Penn. I can ask Chris Penn questions all day long, but not everyone has access to a technical expert on their team to just ask questions. So they’re just kind of winging it and hoping for the best. And so I’m trying to really remove you, Chris, as a crutch of like, well, Chris will. Chris can tell me the answer. Chris can tell me the best way to do this. Yes, I have that safety net, but at the same time, I need to be able to figure it out on my own without you saying, do it this way. Like what you just shared with me about the wiki that hasn’t come up at all during my planning.

Christopher S. Penn: And this is where, and we talked about this in previous episodes of the podcast, being well-read and looking at those edge cases and digging around in places you don’t normally read is so important because you will get vocabulary that you didn’t have previously. For example, when I’m working in Claude Code, there’s this concept in all the AI coding tools called hooks. And a hook is an automatic function that fires at a certain point in the system. So there are hooks for when you start a session, there are hooks when you run a compaction, and there are hooks when you complete an action. And one of the things that is in my Claude Code is I use not a wiki, but I use a knowledge graph. In mine, I have a hook that says post-task hook, update the graph. So I don’t have to remember to update my knowledge graph. I don’t have to remember to update the wiki. The system is built into the system to say, hey, I just finished a task; I better update the knowledge graph so that the next time I look at this, it’s fresh. So part of those requirements would be to say, yeah, the system has to maintain itself. I don’t want to be doing that. The system had better do it for me.

Katie Robbert: But in your example, so you’ve completed a task, what about those external documents, that external context, that external data? Is that part of what gets updated into it?

Christopher S. Penn: Yeah, you have to take notes to say, hey, you’re going to take notes and put the notes in the notes folder wherever your notes folder is in your project, and it will update its notes and say, okay, hey, today it looks like Katie had to nag Chris again about updating his expense reports. And then in the next day’s notes, Katie has to nag Chris again. Day 71. Chris still hasn’t done his expense reports, but it’s in the journal. If we take away all the technical language, it really is just journaling. It’s like every project has a journal, and the AI is responsible for doing the journaling.

Katie Robbert: Okay, all right, offline I’m going to ask you more questions about that. But one of the other big learnings, and this I think was really eye-opening to me but also kind of like a duh moment—which I’m a little embarrassed about, but I’m sharing it because I’m trying to be transparent and accountable—is that a lot of what I’m trying to do and where I think I’m saving resources and environmental impact has nothing to do with AI. I’m using AI to do things that can be done with a script. So for example, I’ve built an App Script into a Google Sheet that pulled from a different API. Like, I’ve built it, I now know how they work. Maybe it wasn’t the most efficient thing, but it worked because I was brand new to it. And we were on a call with a client a couple of weeks ago where the person was sharing this entire process and it was very copy-paste heavy. There was a lot of moving pieces in spreadsheets. And you, Chris, sat back and said, have you looked at the VBA scripts? Do you have access to that? And they were like, well, I think we do. And you’re like, yeah, you just need a script for that. And for me, like the light bulb—if you could see it go off above my head—I was like, oh, damn.

Yep, there’s a lot there that I’m doing that doesn’t need AI to be doing it. And that has nothing to do with cloud-based or local. It has everything to do with being aware of the kind of work that you are doing repeatedly and what is the best solution for it. That for me was one of the biggest learnings in going through this requirements-gathering process, which is why we always stress doing the requirements before you start building anything. So Chris, I’m aware of the App Script and the VBA scripts in Excel, but what are some of the other examples of scripts that people who aren’t overly technical can build and aren’t using?

Christopher S. Penn: If you’re doing this on your computer, which presumably a lot of people are, your computer—whether it’s Windows or Mac—has a command line, a terminal, and in that terminal you can install Assoft Corporate. It allows you lots of little extra utilities. And these utilities are free and open source. Many of them are proven; many have been around forever. And one of the things that I do, which is actually part of my project setup, whenever I set up a project on my computer, this text file goes in it and lists all those utilities that I have installed on my specific computer. So this is not universal to everybody. A lot of these things you would have to install. But this is, hey, AI, don’t invent. If you’re working on a task and I tell you, hey, we need a database, don’t write one from scratch. Here’s a long list of everything that’s already installed on this computer so you don’t have to reinvent the wheel. And these are really like one level up from scripts. These are apps, right? These are text-based apps like FFmpeg, which is an incredible video editing tool. Another one is yt-dlp for downloading YouTube videos and captions.

Another one on here is SQLite Local Database, right? Which is, by the way, once you get started. Doing the wiki thing is a really important tool because it’s an actual database. Pandoc is for converting data from one form to another. There are all these different tools that I have installed on my machine. They’re all free and they’re not right for everybody because you’d want to ask maybe your existing system, hey, for this project, what free open-source command-line applications are the best fit so you don’t have to reinvent the wheel. There are thousands of these things. There’s another one here: ImageMagick. I use this one all the time. Whenever you see my You Ask, You Answer videos on YouTube, all those title cards are programmatically generated by ImageMagick and a Python script. I’m not going to sit there manually typing out every show title. That’s ridiculous. I give the show titles to the script. The script just has a template, fills in the template with text, and spits out 10 or 20 image files at a time. This is where you will find a lot of extra savings.

Another one that’s super important for Trust Insights is the Google Workspace command-line tool. Right. So Claude or ChatGPT whoever can use a terminal script to talk to Google Sheets, Google Docs, and Google Slides without having to reinvent the wheel or use an MCP, which is very resource-intensive and very token-intensive. When there’s a command-line tool that says, hey, here’s how to update a Google Doc, that one is one of my workhorses. I can say, when John sends over a proposal or scope of work, okay Claude, you’re going to use the GWS CLI to read this doc and here’s my criteria for how I want you to review it. It will now add comments to this doc, redline this, or whatever, and it’s all built into that tool. Zero tokens, zero AI because it’s just a command-line tool.

Katie Robbert: This breaks my heart because I fought learning command lines for so long, and yet if I’m really committed to reducing my use of AI where it’s not appropriate, then I really do have to learn them. And so I will go on record and say I’m going to start exploring the command-line apps that make the most sense for me based on the work that I’m doing, which is what is coming out of my requirements gathering. And so I have a very strong use case where I had the methodology clear and I was generating report after report about more than a dozen of them. They were all identical, just with different data inputs. And then I had to change something in every single one of those reports, and I regenerated them again. And when I was doing the requirements, it very clearly flagged that should have been a script. Why are you using AI for that? And it’s just… For me, it was just my lack of education around what was possible. But now I’m looking at everything that I do and I’m like, could that be a script? Could that be a script? Could that be a script? Could that be a script?

And that is the third category of how to save on AI use that I wasn’t expecting. I was looking at it very binary: local or cloud. And that was it. And now there’s this whole other category that I’m like, well, that just changes the game altogether.

Christopher S. Penn: This is something I wrote about on LinkedIn not too long ago. One of the best uses of AI is to make tools, scripts, or applications so that you don’t have to use AI at all, because these tools are the best coders in the world. And as long as you’re good at following the SDLC, you can say, we’re going to make this piece of software, and then I’m going to install it on my machine and then I’m going to use it. Instead of AI, I’ve got one tool that I use called File Prepper. All it does is take a directory of text files and glues them all together in one big old text file. There is zero reason to use AI for that. And it puts metadata that the output file can then be put into AI, and AI can read it and go, oh, I understand this file system layout because there’s all the metadata about the file name and all the stuff that the script pulled together. But I’ve probably got a dozen tools that I’ve built for myself just in the work I do for clients that are scripts where there’s very little AI, or there’s AI that does only the thing that AI is good at.

So I have one script that takes a PDF and turns it into a text file. But what’s different about it than all the other tools in the market is that it calls a vision language model, a local one on my computer, to say, there’s a diagram in this PDF. I can’t translate a diagram. So, hey, vision language model, look at the diagram and describe it. And the description gets entered in the text file. I can have it read a technical paper and it will put a system diagram in text in its markdown. That does not exist right now, at least when I look for it. So I’ve said, okay, build it.

Katie Robbert: What strikes me about all of this is all the companies that are marching forward saying, use more AI. We want more usage. We want everybody using AI. And here I am against one person, one sole project, and I’m learning so much in a short amount of time: you don’t need as much AI as you think you do. And it’s not putting you ahead of the game. And to be fair, yes, at Trust Insights, we consult with our clients around how to use AI appropriately. But we’re also the first ones to say you don’t need AI for that. In that client call that we mentioned earlier in this episode, we were very clear: you don’t need AI for that. You could use AI to help write the script, but the execution of the script over and over again for all of these manual processes that you have? You don’t need AI for that. And I don’t think that paints us in a bad light of, well, they’re supposed to help me use more AI. I’m not going to sit there and tell you need something that you don’t. And Chris, you’re definitely not going to because that’s when big, expensive mistakes happen.

I saw what I hope was a parody video on LinkedIn, could have been probably not based in reality. It was basically this like executive holding their laptop coming into the conference room, ramming into the conference room, yelling about bills and blah, blah. And he’s like, I thought we were using tokens. Tokens are made up things. And this poor woman sitting there is like, tokens cost money. And every time you use AI, it costs tokens. Like, right? So why am I getting bills? She’s like, because that’s how it works. He can’t wrap his head around the fact that every time he uses AI, it costs money. And yes, the tokens feel like arbitrary usage because the machine is just deciding how much gets used. But at the end of the day, it still costs money. And then for us, where I started was with it taking resources from the environment, using water to cool the data centers, and there’s more data centers going up. So it became this whole really big thing of trying to understand how to not contribute to bad environmental things. And a lot of what I’m learning is it has nothing to do with AI. We’ve just become so reliant on AI being able to do it for us. Because it can doesn’t mean it should.

Christopher S. Penn: You know, we always—I always use the analogy of it’s like blenders. There’s a time and place to use a blender. Like when you don’t want a nice frozen margarita, you’re trying to cook a steak. There’s no place for a blender in the actual cooking of steak. Unless you want like steak gel. Right.

Katie Robbert: I mean, someone might—there might be—but this is sort of those edge cases that you talk about where every once in a while there’s a use for that, but it’s not the most common set of use cases.

Christopher S. Penn: Exactly, exactly. I guess supposed to be. What do they call it? Mechanically extruded meat, I think is the name. So it’s whatever the term is. When you look on the chicken nuggets packet, says mechanically separated protein.

Katie Robbert: I mean, where’s my… The more you know about rainbow [orders], I don’t. Where’s your red flags?

Christopher S. Penn: Right, it’s over there. It’s not in reach. I haven’t had to listen. I haven’t needed a red flag in a while, and that’s a good thing. Exactly. But yeah, and there’s even technological layers. So one of the ways to tell—and think about the types of AI to use—is the categories of AI that are available to you. Right. So you have big, heavy foundation models, you have workhorse models, you have lightweight reader models. And then you have closed-weights tech providers, you have local inference providers like DeepInfra or Cerebras. And then you have literally on your hardware. So there’s almost kind of this panoply of menu items of what kind of AI do you want? And the hardest part for a lot of people right now—we were talking to somebody earlier last week—was they were saying, I don’t even have time to read your Prompt Playbook subscription. Right. So that’s a person where the biggest, heaviest use of AI is probably not the best-case use case for them because they’re not using it now.

Katie Robbert: So for me, my next step is to really start to pick apart the PRD that I’ve put together so far and really challenge myself in thinking, have I really thought through enough about using apps or scripts versus is this still just local or cloud-based? So really trying to deconstruct the work that I’m doing into how much of it really can go into a repeatable script that lives offline from AI altogether because I’ve acknowledged it in the PRD, but I haven’t really dug into it enough to say with confidence this is what this is going to look like. And this again is why we really recommend going through requirements before building things. Because I would have already been halfway into some sort of a local migration. I probably would have bought a whole new laptop, which costs a lot of money. I probably would have spent a lot of time trying to set up a local environment that honestly I wasn’t going to use as much as I thought I would, and I’d still be relying on the cloud-based version because of the ease of use for myself.

Christopher S. Penn: Yep. And as you get further down this path, we’ll probably end up talking about some of the different values to be had in terms of where you’re going to spend your money on AI. Because what’s happening in the technology is there are faster, cheaper, and better solutions than what some of the big-name brands would like you to believe. And so that’s a part; that is part and parcel of it. But one of the things that I’ll put a bug in everyone’s ear to remind you to think about is if you look at the API costs of any model or service, the more expensive it is, the higher the impact it is. Because obviously, a super heavy compute model costs that provider a lot of power and water. They pass that on to you in cost. But look around, and we can even do this like on a live stream or something sometime. Look at the cost, the price of a model versus its intelligence. Because there are some really, really incredible models out there now that are high intelligence, very low cost—though not the brands you would expect to be at the top of the list. And those if you are in a situation where from an environmental perspective you’re like, well, do I really want to go buy $7,000 worth of hardware to save $80 bucks a month? Or is there a middle ground where I can use a fast, intelligent model that is low impact because it’s low-cost?

Katie Robbert: With the huge asterisk that’s not going to be available to everyone depending on the professional environment you’re in. If you’re doing this for your own personal use, it’s going to be great. But if you’re in an environment that is locked down in terms of what you can use and what you install… big old disclaimer that what we’re sharing may not be available to you. And that’s an argument you can bring to your C-suite, to your IT department about cost savings and environmental impact.

Christopher S. Penn: Exactly. So if you’ve got some thoughts about how you are balancing what kinds of AI and whether AI is even the right choice to solve a problem, and you want to pop by and share them, stop by our free Slack group. Go to Trust Insights AI analytics for marketers, where you and over 4,700 other marketers are asking and answering each other’s questions every single day. And wherever as you watch or listen to the show, if there’s a channel you’d rather have it on, go to Trust Insights, the TI podcast. You can find us at all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch, and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology and Martech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL·E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMO or Data Scientist to augment existing teams beyond client work. Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights podcast, the Inbox Insights newsletter, the So What? livestream webinars, and keynote speaking. What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data storytelling. This commitment to clarity and accessibility extends to Trust Insights educational resources, which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI. Sharing knowledge widely, whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 03:15 – The challenge of information overload 07:40 – Why AI defaults to common advice 12:20 – Deductive versus inductive thinking 16:50 – Finding psychological safety for exploration 21:10 – Cross-pollinating ideas from other industries 26:35 – A practical research workflow 31:00 – Call to action

Hit play to uncover how shifting your perspective transforms stale ideas into bold marketing breakthroughs.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights. This is the age of AI, where there is more information than ever, more data than ever, more everything than ever before. And yet behind the scenes, when we were prepping for this episode today, Katie, I said, hey, what would you like to talk about today? And your response was, I have no idea. Let me ask you a couple of different questions. One, is it that the material recovering feels like it’s been done before, or is it also that there’s just so much to choose from that you end up with what Barry Schwartz calls the paralysis of choice?

Katie Robbert: Yes. So let me pull those apart. So much is changing in the field of AI and where we focus down in on with analytics, marketing, operations, and helping businesses. We do such a good job of thoroughly exploring topics, and the process that we have for creating content is so efficient that we can churn out a lot of it very quickly. So I feel like personally, the things that we would cover that are relevant to our business and relevant to our audience now there’s always another stone to turn over. But then you start getting really deep into the weeds. And where I wrestle is do you start alienating parts of your audience to really focus on those very specific niche pieces? To the other part of the question? Yes, I do feel like there’s so much information. Again, this is sort of the side effect of generative AI that you sort of become numb and blind to all of the information because there’s so much. I remember when we worked at the agency and you used to produce this report, I think it was yearly, on the number of pieces of content created. And you know, this was prior to generative AI, this was just the number would go up exponentially year over year. And you know, I can’t even imagine what that number would be now. And just sort of seeing the uptick. And so I don’t, I feel like I’m deep in the weeds of stuff that I’m working on day to day in the business. Part of it is I’m like, I don’t know if anybody would even care to hear about this, if it even resonates or if it makes sense or if we’re supposed to be doing that big thinking and like introducing new concepts and you know, doing the thought leadership of what’s happening with AI and analytics. Here’s the thing about analytics. There’s only so many ways. There’s a lot of ways to analyze data, but there’s only so many ways to analyze data. And so I feel like at this point, unless a brand new analysis methodology is invented, which I could happen, but I feel like we’re pretty good, we’re solid, you know, it’s not worth talking about again. But I could be wrong. I could just be having a very blah Monday morning where I’m like very feeling very existential of like, what’s the point of any of it?

Christopher S. Penn: Which, given the macro picture, which is a totally separate podcast, is true. In the catalog of analytics that we use, that I use, there’s about 1400 techniques that span 22 disciplines, everything from weather forecasting to agronomy and agriculture and stuff like that. And one of the things that I was talking about this weekend in my own personal newsletter was how limited marketers are in their choice of analytics when you use AI, because AI naturally gravitates to what we’ve already written about. So we’ll talk about attribution analysis, we’ll talk about social media marketing, we’ll talk about impressions. Our advisor, Ginny Dietrich’s most hated measurement of all ad value equivalents. And these are all associated statistically, probabilistically in these AI tools with marketing analytics. So if you say, hey ChatGPT, I need some marketing analytics help with this, you know, Google Analytics data source, it will give you the most probable things which will be old hat. Like, it will be like, oh, have you tried doing, you know, basic trend analysis? Like, yes. Have you tried doing time series analysis? Yes. Right. And that is a blind spot that AI can never overcome because of its nature as a probability based tool. You, the human, have to have the largeness of vision and the vocabulary to say, hey, you know, chat GPT or co pilot or whatever. I’ve got this issue here and I’m having trouble understanding what’s driving the set of changes. What analytics techniques from agriculture and agronomy might be a good fit for this data and suddenly be like, I know all about agronomy, which it does. And have you tried pairwise combination holdout? And you’re like, what is even? I don’t even know what those words mean.

Katie Robbert: Nope. And I hear what you’re saying and I feel like we’ve talked about versions of this where in order to get the most out of the tools you have, you really have to be curious and thinking outside of the box and that’s something, Chris, that you do very well. That’s one of your strongest professional attributes is you say, where can I look outside of this lens and see like and pull out like agriculture. So I don’t think that’s the default thinking for most. Take marketing out of it. For most people, it’s like I’m looking at a problem. I’m trying to figure out why my bread dough isn’t rising. Let me look at the fashion industry and see what I can learn from that. About, like, it doesn’t make sense because it feels like putting square pegs in round holes and I don’t know, maybe that’s sort of where I’m stuck, is maybe I’m not someone who’s good at looking outside of. Like, I’m a very rules and process oriented person. I like things to be predictable. So perhaps for me, that’s where I struggle with this concept of I’m trying to analyze my marketing data. How does agriculture do it? Like that to me is just like, why would I ever do that? That seems ridiculous to me. That’s not how this rule works. And so that’s definitely not one of my strengths. Which again is why you and I are very complimentary in our strengths and weaknesses. And you know, I just went. When we come back to the purpose of today’s podcast, I feel like, you know, maybe that’s where thinking about topics to talk about is where I’m struggling because I feel like I’m a fairly creative person. But I also, I remember I started taking art classes in first grade and went all the way through 10th grade of high school. And I, my skill set really didn’t progress. It really wasn’t something that, like, it was something I wanted to be good at. But I just, I’m not an overly creative person. I’m creative when it comes to problem solving. But I feel like I also have a really Good toolbox. But if you start to say, okay, your toolbox is now going to contain these things that belong in, you know, biochemistry and welding, those are not industries I know a lot about. I’m like, nope, I don’t want that in my toolbox because I can’t, I don’t understand it. I don’t get it. I don’t know. There’s a lot of like therapy for me this morning.

Christopher S. Penn: Well, what you’re describing is the difference between inductive and deductive reasoning. So as a rules based person, you would be a deductive reasoning person. You have a set of rules and principles and concepts in a domain that you break down to specific conclusions. If we think about the 5P framework by Trust Insights is exactly that. Five different areas, purpose, people, process, platform, performance that have governing principles for each of these. And you could take that general thing and distill it down for any problem that you run into, right? And say, okay, how can I make this. We’ve you’ve talked about even making it a prompt framework for AI because it’s so good at it. The opposite is called inductive reasoning where you say, I got a bunch of stuff, it’s all very confusing, it’s all not related. What is in common? And so for example, if you look at agronomy and weather forecasting and econometrics and marketing, what is the through line, the weird thread that goes through all of them for marketing analytics, and that is math, right? The mathematical techniques that each discipline has to use that are different from discipline to get to the answers. And so inductive reasoning says, can we find those threads from these crazily not related fields, hold on those threads and turn them into something like, oh, we can use this in marketing. We have data that resembles this. How do we adapt it? And so that’s the difference. And so when we look at like content creation, saying what are we going to talk about today? If we start from that deductive reasoning perspective of, well, we’ve done AI 18 ways to Sunday. We’ve done prompting, you know, 22 ways to Sunday. Well, they’re left to do that is a deductive perspective. And there’s, that is a great way to do it because you cover everything. The inductive way is okay, say okay, this is what you flip to when you’re like, we’ve done it all, what’s left? Okay, now you flip to inductive reasoning. Say what about this weird edge case over here? Let’s pull that apart and see where it leads.

Katie Robbert: Well, I mean, and this aligns with what we always joke about, that you know, what will be written on your tombstone is what does this button do? You know, and there’s a part of me that wishes I was a little bit more open, more curious, you know, I feel like I am, but at the same time, not nearly as much as I could be in terms of, you know, in this context of like breaking out of what I know. And so what kinds of advice would you give to someone like me who, to your point, like, if I do the deductive reasoning and I cover all the different known angles, you know, and I think about things from a risk averse lens, what advice would you give to someone like me to then find ways to turn on some of that inductive reasoning when I need it?

Christopher S. Penn: You can generally teach inductive reasoning in a couple ways. The one of the things, and this is very important, what you just said, you said from that risk averse lens, that’s the first switch that has to flip is to say, okay, you can do exploration if you can cycle, give yourself a sense of psychological safety that you’re not taking a risk. Like going out and reading random stuff on Reddit and things or people you don’t Normally follow on LinkedIn is not a risk. It can feel like a risk because it can feel like distraction, it can feel like confusion, but it’s not a risk. And so, one of the things you have to find for yourself, I don’t know how to do it, is to say, okay, let’s create a sense of psychological safety that says it’s okay to go look under this, under the stone. You may not like what’s under there, but it’s not dangerous to you. And the second thing is those fringe edge cases where, and this is where, like, LinkedIn is a terrible place to spend time because everyone is stuck in the same bucket, right? We’re all talking about, oh, AI is going to do this. Sam Altman did this. And it’s the same theme over and over again. You have to leave that and go to the fringes, go other places. Reddit is a good place because there’s, it’s a different set of fringes. Talking to our friends and colleagues, right? Chatting with them in group texts and stuff is a great place to do that, where you start to hear very specific edge cases. You’re like, oh, we’ve never run into that before one of the things. So it was really interesting. I did a drawing for my newsletter for filling out a survey back in July. And the giveaway was a one hour. Ask me anything private. And the conversation I had with the person was an industry that we don’t work in at all, like heavy industrial marine stuff. And just. And I learned as much as I talked, like, oh, I didn’t know that was a thing. And those, all those weird little edge cases was like, that’s interesting. And so that might be a possible antidote too, is to say, like, you know what spend. Say, I’m going to spend an hour talking to this person over here. I. Who we’re probably not going to do business with them. We’re pro. I’m. I’m not there to drum up business. I’m there to say, what kind of weird stuff are you running into that maybe wouldn’t cross my desk.

Katie Robbert: H. I hear you. I think, you know, I like the idea of, you know, doing more reading outside of what you’d normally read. I agree that LinkedIn is very homogenous, very bland. The challenge. And, you know, this could be again, my own personal bias and my, you know, I like the term that emotional safety. Because if I’m being honest, that’s a big part of it is it’s not that I don’t feel safe talking about things outside of what we’ve talked about. And perhaps this, you know, this is all just sort of like a, you know, a horse of a different color is that I get concerned that if we stray too far from what our audience wants to hear about, they won’t come back. And that’s, you know, that could be my own personal concern and anxiety about it. And maybe they’re like, no, give me something different. You know, there’s really only one way to find out and that’s to ask them. So if you want to pop over to our free Slack community analytics marketers and let us know. But to your point, you really. What I’m hearing you say is it’s breaking outside of those comfort zones, those routines, which admittedly is not something I am personally great at. I’m very introverted. But reading outside of what I normally read and interacting with people online is not so uncomfortable that I can’t do it. I just need to make more of an effort to do it because then it’s, you know, it’s making sure that we’re staying in touch with things outside of our own little bubble. It’s a big reason why we have the community that we have. So know a little bit of tongue in cheek like join our free Slack community but at the same time join our free Slack community because we need it just as much as the people who are joining need it. So that we’re not just locked into well this is what we’re doing. This is how we’ve always done it. So this is how it shall be. End of sentence. Like I think that by asking the question of the day it that’s partly to understand like what’s going on in your world, what are you dealing with, what are you, what’s your perspective? And I’m always not. A question goes by and we ask them every single day that I’m not like huh, that’s interesting or I never would have thought of it that way. And I think that’s such an important thing. But I never thought of that in this context. Chris of that inductive reasoning of trying to find that sort of like through line through different topics.

Christopher S. Penn: I will show, I will share with you an interesting observation about our analytics from Market Slack group. There’s a, there’s a topical split by gender. When you look at the questions people ask, even that itself is interesting. You know the, the people who are male identifying tend to be asking very technical questions. Hey, I’m doing this with this agent framework and you know, what’s the best? And the people who identify as female typically asking about outcome based things like you know, what’s better for getting this done. I need to get, accomplish these things. And so one of the things because obviously, you know, equality in general is important to us, but gender equality is very important to us as a company. You might want us to do privately, quietly reach out to some of the individual women in our community to say like hey, I’d love to have a one hour private, ask many things, I just want to know what’s going on. Talk to the folks who are already reasonably talkative in the community and just see what’s happening at the edge cases of their Worlds. You have Dr. Leslie who spends a lot of time on psychology and research. You have Hannah, you have Minion who spends a lot of time on writing in AI. And that’s where you can be reasonably assured of a psychologically safe experience and still get some really cool edge cases.

Katie Robbert: Yeah, I think if I’m being honest, I would probably start with Reddit forums or something like that just because that’s for me the path of least resistance and doesn’t take extra coordination and time to figure out how to, you know, book this conversation with someone. But again, that’s all, you know, minor details, but. In general, you know, so let’s say every week we sit down to record the podcast. And every week I know you’re going to ask me, hey, what do you want to talk about this week? I mean, that’s just how it goes, you know. And I appreciate that you give me the opportunity to propose a topic versus it just being, you know, led by whatever you’re reading that week, which often happens because I’m like, whatever, sometimes I have a topic. You know, tell me a little bit about your process for like you mentioned agriculture and some other industry that I’ve never heard of. And I’m guessing I haven’t had a chance to read your newsletter yet for full disclosure. No, I mean, and I do. It’s on my list for later today. Is it more that like you look at like a list of something, you go, I don’t know about that. That’s interesting. Let me learn more. Or is like does it sort of organically just come up because you’re like searching for something and it just sort of leads you down a rabbit hole? Like what does your process kind of look like?

Christopher S. Penn: It’s both. It is both. This past week though was on idea transfer. So I ha. I shared some prompts, some basic prompts to look at four level naics codes which are like profession codes, oil and gas extraction for example. And I said to the soft to I was using deep seek for this. Go and find the professional academic conferences for each of these professions out of the 340 professions and then find the award winning papers from 2025 and 2026 for each conference if they had one, and end up with like 300 some odd papers that all won gold medals at their events. And so download those papers and then start extracting out what are the key lessons that we could transfer into marketing and marketing analytics, especially from these award winning papers. And some of them there wasn’t, you know, some of them it was like a position paper. But there were a lot of very technical research studies like oh, that’s really cool. Like how the European. I forget what the organization name is, but it’s the European Council of Weather Forecasting and their new neural ensembling system for better weather forecasts. I’m like, that’s clever. Instead of trying to fit a neural model on an overall forecast to fitting neural models on individual components of forecast because they can forecast better those specific components and then glue the forecast together instead of trying to ensemble the whole thing. It’s like, you know, instead of trying to. People screw this up a lot. Westerners tend to try and make fried rice all in one big wok and your home stove can’t get that hot. So you cook each set of ingredients individually and then you mix it all together. That’s this, essentially this version of ensembling. Don’t try to make fried rice all at once. Make each of the components, get them each right individually and then you can just literally mix it all together.

Katie Robbert: Well, at least I know I’m making fried rice correctly. Thank goodness for that. I’m taking the win today. But that’s an example. So the topic was how would you make it a. A fully AI written newsletter with no human intervention whatsoever that would still be valuable. And so I had it to a test run and the example was that pairwise combinations for agriculture. I read the newsletter of IT generator. Like crap. This is a really good idea. Like I gotta go figure out how to do this. I was reading the results as I was recording my, the newsletter on video. I’m like, this is really smart. Like it should be. It won an award. But like I can totally see how this applies specifically to social media marketing.

Christopher S. Penn: Interesting. Which comes back to you have to have expertise in something in order to be able to, you know, apply it. So you know, I know what I know about running a business. I know what I know about operations and process and all that. So it’d be interesting to see what I can find outside of the lanes that I know, you know, because it tends to feel like, well, process is process. There’s only one way to do process. And I’m assuming that’s just not true. And so I need to look at what that could mean. I have no idea. And I think this is where my risk averse brain starts to kick in. I’m like, well if you have no idea where you’re going, you don’t have the time to waste to just sort of start exploring. You have other things that you have to do to keep the business running. And I think that’s just sort of a time and resources but also a mind shift thing. Because yeah, I have to keep the business running. Do I have the luxury of time to just sort of do some exploration? I don’t know. I’m hoping to find some because I feel like that would make me a better leader if I could.

Katie Robbert: And you’re like, yeah, you could be A better leader. Way to go.

Christopher S. Penn: Everybody has the opportunity to better. You could, if you wanted to and you didn’t want to spend a ton of time on it directly, you could purchase. Well, you know, you don’t have to purchase it, but yeah, the new deep research suite from the Trust Insights Academy, which helps automate a lot of the research process. And where I would suggest that you start is in the epidemiology around opiates and stuff like that. Because you used to work in that field. You know it like the back of hand. But it’s been 20 years. Yeah. What’s, what’s happened? What papers have come out in that field in the last even five years that, where the math has changed, where AI has changed that field. And you can go, oh, I know how we used to do it and I know how AI does it now based on these award winning papers. Holy crap, things are different. And I can take this, what I know from back then and what is steady state now, current state now, and go. I can apply that learning of how AI has changed that field to what, running our business.

Katie Robbert: Yeah, that’s interesting. That actually already gives me a couple of places to start because I think about a few of the clinical trials that were running and they were so manual because it just was, it was just manual. That’s how it was 20 years ago. And so like hand coding things and human raiders and all of that, which at the time was fine. We didn’t know any different because there wasn’t anything different. This morning we had a lead file of 150 leads that needed augmentation and I had an AI go in and label it all and it took under 20 minutes. And I ran it locally. It didn’t have to use the cloud at all for it. And so that’s today. Yeah. It reminds me of when I was trying to do lead gen for the sales team at that same organization and I stumbled across a publicly available database of substance abuse clinicians and it needed cleaning up and it needed like all this stuff and it was just a manual process and it took me a couple of weeks to do it before I could even hand it off to the sales team. And then those ungrateful jerks were like, what do you mean there’s not an email address? And I was like, you got a phone number? Pick up the phone. What do you want from me? But anyway, yeah, it is definitely interesting. So I think I don’t do that often enough. I really try to stay grounded in like what’s right in front of me. And I think that maybe looking in the past of like what I had done, what I have accomplished might help at least get some of this started.

Christopher S. Penn: Exactly. There’s a lot of. There’s a lot that’s changed. But you having that previous subject matter of expertise means that it will be easier to dip your toes in that specific angle on that discipline rather than going to like agronomy where you’re like, I don’t even know what any of these words mean because like what is this? This yield, you know, yield per field thing. If you’ve got some thoughts about how you are approaching deductive and inductive reasoning concepts for content creation or ideation, pop on buyer free slack. Go to TrustInsights AI analytics for marketers where you and over 47100 other marketers are asking and answering each other’s questions every single day. And if there’s a channel you’d rather have this show on that instead, go to Trust Insights AI TI Podcast. You can find us at all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence and machine learning to empower businesses with actionable Insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence and machine learning to drive measurable marketing roi. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology and martech selection and implementation and high level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google, Gemini, Anthropic, Claude Dall E, Midjourney, Stable Diffusion and metalama. Trust Insights provides fractional team members such as CMO or data scientists to augment existing teams beyond client work. Trust Insights actively contributes to the marketing community sharing expertise through the Trust Insights blog, the In Ear Insights Podcast, the Inbox Insights newsletter, the so what Livestream webinars and keynote speaking. What distinguishes Trust Insights in their focus on delivering actionable insights, not just raw data, Trust Insights are adept at leveraging cutting edge generative AI techniques like large language models and diffusion models yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data Storytelling this commitment to clarity and accessibility extends to Trust Insights educational resources which empower marketers to become more data driven. Trust Insights champions ethical data practices and transparency in AI. Sharing knowledge widely whether you’re a Fortune 500 company, a mid sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 02:15 – The AI writing frustration 06:30 – Measuring your voice 12:45 – The five performance steps 18:20 – Building your writing blueprint 24:10 – Call to action

Take the new course at: https://academy.trustinsights.ai/courses/ai-for-writers

Watch this episode to finally break free from generic AI output and start writing with total control.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights, let’s talk about AI for writing and for writers. We have seen no shortage of people talking about AI watermarking and all this stuff and how you can tell whether somebody’s using AI for writing or not. And we at Trust Insights have put together a new course, Trust Insights AI for Writers, for how to get AI to write better and not coincidentally, help you as a human also become a better writer. So, Katie, to start off, what are the things that when you are writing with the assistance of AI, what are the things that sort of you wish AI would do better?

Katie Robbert: You know, I wish it would listen better. And by that I mean I feel like you can craft a really strong prompt. You can say, here are my writing samples, here are things that I don’t want you to do. And it kind of just like freewheels and does its own thing anyway.

And I feel like that is frustrating for a lot of people. So, you know, I really try to write the first draft of things myself as the human and then know, and I’ve talked about this in the newsletter and on pod on our podcast where I bring in AI is to double check with like our ICPs or to use it as an editing tool. But then the editing gets carried away.

You know, I’m not an editor and grammar is. I would like to say that the public school system failed me. You know. So I’m like a half decent writer. I have good ideas and I write in a stream of consciousness. And so I need tools or human editors to help me clean things up.

And this is where I look to generative AI because the team doesn’t always have time to like fully edit my stuff. But then when I read back what the edits are or what the suggested edits are, I’m like, where did this come from? Or why did you make up a whole anecdote that never happened, but you’re saying it authoritatively?

So I feel like the hallucination is the big thing. And the, you know, depending on the large language model, each model has its quirks in terms of the way that it writes or the way that it, you know, quote unquote, articulates thoughts.

And so I think one of the things that you’ve shared about Claude, for example, is it likes its, you know, things in threes, like three punchy points. And it’s very much a tell that, like, oh, that’s a Claude thing. And I found that when I have, you know, an AI assistant help edit my stuff, it turns it into a lot of that, like the individual sentences, the three punchy points, like the this and this, then this or this or, you know, those kinds of ways that it writes.

And that’s not what I, the human had put in. And I’m just sort of sitting there exhausted, like. But I just, I need this edited and I need it coherent and is it good enough? But did it change too much of my own human writing? So I think that, you know, when I think about using AI for writing, that’s what I’m personally struggling with is, you know, I still want to be the one writing it, but, like, I need someone to help me polish it. And the polish is like subpar.

Christopher S. Penn: Why do you think AI does that? Why do you think AI behaves the way it does and turns original writing into something that sounds like slop?

Katie Robbert: Oh, gosh, if only there was a course that was going to tell me the answer to this question.

Christopher S. Penn: There’s something else that will also tell you the answer to that question. And that happens to be the fifth P.

Katie Robbert: I should have guessed that one. Sneaky, sneaky. The framework at Trust Insights is purpose, people, process, platform, performance. Chris, you’re specifically talking about performance. And I think that where a lot of us get caught up in prompting these large language models is we think we’re being clear on the performance, but we’re not as clear as we could be.

And that’s where the frustration sets in and that’s where we want to throw up our hands. And so the perfect purpose could be, I need you to edit this, you know, five thousand word essay. I need you to look for spelling and grammar and, you know, a cohesive thread like all those things. People, here’s my audience, here’s my authoritative voice, here’s my samples process. I want you to go through this and just list out the changes. Don’t change it for me, platform. This is going to be published on my blog, which is hosted here, and I’m going to add images in these places.

And then performance. We typically think of performance as did we get the polished thing from our purpose? But it sounds like we are missing a lot of opportunity in the performance part of the 5Ps to really spell out what we need.

Christopher S. Penn: And that is the premise of the new course. The biggest chunks of the new course that we have are twofold. One, we spend a lot of time on research because good research leads to better outcomes typically. And two, we spend a lot of time on math, which is your average writer is like, But I started with the premise for this course, that writing is code.

If I put nonsense words together, you’re like, did you just get hit in the head? Like, what happened? Did you actually put decaf in the coffee maker this morning? If I don’t say words in the right order in a statistically predictable pattern, you have no idea what’s going on. You might say, these tests, coverage, adding branches, empty. Like, what? What does that mean? That’s word salad.

Language follows patterns, and those patterns are predictable. And the reason why AI writes the way it does is because it’s choosing the most probable patterns, even when it doesn’t sound like you. So the first thing that we have to do is give AI performance, right? To say, this is what success looks like. And it has to be in a tangible form.

The percentage of passive voice that you use in a text when you write as a human, how much passive voice do you use? The number of sentences that begin with a noun or a pronoun. What percentage of your copy is that? Is that the number of EM dashes that you use naturally in your text as a human? What is that? Our friend Anne Hanley says, I use the EM dash because I’m an actual writer, but I don’t use it in every sentence.

And where AI typically goes off the rails is when it knows that a construction is probable, like using EM dashes, like using triadic rhythm, like using bicolon or isocolon. And it says, hey, I’m going to use the most probable things. But it has no concept of frequency, so it overuses it. And you get, it’s not this, it’s that in every single sentence or in, you know, Claude in particular loves bicolon. It’s. It sounds like a drum beat. One and two and one and two. And you’re like, could you please vary the beat?

Katie Robbert: Right?

Christopher S. Penn: When you look at a, like a slide deck Claude generates, everything is bicolon, all the headings, you know, this and this, sharp insight and this. And you’re like, my God, this is so mind numbing to read. If we give AI analysis of how our writing to begin with and say success looks like this set of numbers, now go right, then check your work and compare what you wrote versus what the blueprint is and it will go, oh, I didn’t do this at all. Like, I use 82% passive voice. Yeah, go back and fix it. But the fifth P in the 5P framework by Trust Insights is so important. We have to establish what success looks like for it so that it mathematically can go back and fix its code.

Katie Robbert: So let me ask you this question though, because I will give Claude like samples of my writing and say this is what it’s supposed to sound like. Am I doing it wrong? Because I have all of these samples from like literally years and I feel like Claude or a large language model, you know, is trying to evolve my writing so that my writing fits its format, not its editing, to my writing. Like, I feel like that’s where I’m struggling.

Christopher S. Penn: You are not doing anything wrong except you are asking it to count and it can’t count. And so even though it will analyze your writing as a language model, it has no clue of how to count. One of the things that’s in this course is probably worth the price of admission alone is a Python script that it has to run on a writing sample you provide that will do that fingerprint mathematically not letting the language model try to count, because language models can’t count.

The Python script goes through and it counts in your original sample, this is the percentage of passive voice that you use and it writes it down as a fingerprint, as a file in your language model of choice. It works in ChatGPT, it works in Copilot. It tested in all the systems. It can then rerun that script on its output and say, initial sample, 4% passive voice, my work, 18% passive voice revision loop. I need to go back and keep revising until I hit this number.

But it can’t count that by itself. It needs the support of actual code to do it. And that’s what’s in the course is pre baked. Nobody has to be coding, no coding involved. It’s bundled in. But you would drop that into your Copilot or your ChatGPT or your Claude and say, this is how you’re going to measure yourself. Yourself, you’re going to do the fingerprint and you’ll reuse that fingerprint and then you will go back and you will count using this script. And that’s, you know, again, you’re not doing anything wrong. It’s just the average non technical user doesn’t think, hey machine, I remembered you can’t count well.

Katie Robbert: And I say, okay, so that’s an interesting distinction because so up until now, you know, I’ve been saying like, hey, this is the sample of my writing. And you’re right, it absolutely, it takes what I give it and it like, is like, oh, you said to do this, let me make sure that I do this. And by making sure that I do this, I’m going to do it 16 times out of the 18 sentences, but I’m also going to do this in 15 of the 18 sentences. And so it’s taking everything it knows about my style of writing and trying to jam it all into one sentence. And I’m like, whoa.

Like, yes, grammatically it’s correct, but now it’s garbage because that is not at all what I wrote. And I sort of. I feel, it’s not that I feel like my hands are tied because I can’t fix it myself, but like, the reason I turned to a system for help is because I’m not an expert editor. And so I miss things that an editor would find. And I need that, like, for me, I need that kind of support of like, hey, you started a point over here and then you dropped it halfway through and you made a different point at the end. Like, you gotta pick a story and stick with it.

Christopher S. Penn: And this is where these tools, these AI tools simply by themselves cannot do that. Like, they just do not understand. How do I, how do I even count? So I’ll show you an example of one of the things that is bundled in the course. And again, the average user does not need to look at this. The average user is not going to. You will just drop the file and say, machine, off you go.

But it will look at thing. There’s functions in this. It says like, look at the rate per word of the this kind of word. For example, it’s often said that good writing relies on relatively few adverbs. Adverbs are words that end in ly, you know, actually, etc. AI loves adverbs, obviously, actually, and stuff like that. Which also sounds condescending.

Katie Robbert: Yeah.

Christopher S. Penn: And so if your writing style uses almost no adverbs, when you do a fingerprint, it will say, hey, your adverb rate per 1000 words is like 1%. And so when it goes back and counts, it’s revision, it’s edits that you had it make. And it goes, oh, I used 9% adverbs in my revision. But the target, the performance says 1%. I need to go back and fix my work. It’s a diagnostic. And that’s what’s missing from all of our prompts, because it can’t do that.

That’s what’s missing from every single AI for Writers course I’ve ever taken or every session I’ve ever sat in. Nobody thinks of writing as a system of measurement, of analytics. And therefore, when AI just follows its own internal process programming the probabilities that it generates, we’re all like, why is this keeps coming out like slop? Why can I not prompt this thing to sound like me? It’s because it can’t count.

And so the cornerstone really, of this entire course that we’ve created is let’s give AI the tools it needs to count, let’s give it the measures that it needs to count, and then let’s give it clear guidelines. You know, one of the things that is in the toolkit is a. Again, this is another piece of code the user, you. The user will not use. This is built into a skill that you just install, and there’s instructions on how to install it. But it will say, if you have this cadence, don’t do that. Here’s how you do it instead, right? If you do short, long, short, long cadence over six consecutive sentences, you’re writing us. You’re writing a dead drum beat. Dead. Don’t do that, do this instead. Things like that.

So, for example, if you look at AI writing ChatGPT, Claude, Gemini, and you just take a step back and you look at the page, all the paragraphs are about the same length. They’re all. They all kind of look like the same gray rectangles on a page. If you were to step back and stop looking at the letters and just look at the shape. If you look at your human writing, there’s a good chance that it’s. There’s some paragraphs are real short. Maybe it’s even one word, like an emphasis one like, no, don’t do this. Right? And that’s just its own paragraph.

That frequency of change in paragraph length is something that you can measure. Machines don’t know to look for that. Machines don’t even think about that. And so if we give them the tools, the counting tools to go. Katie tends to alternate her paragraph length. Sometimes her paragraph length is 11 sentences, other times it’s one. I should replicate that general pattern. Because once you give AI a pattern, it’s like, oh, I know how to do patterns. I can do this. And it goes off and it creates it.

Katie Robbert: I mean, I have a lot of thoughts and comments, you know, and so like, the big elephant in the room is, you know, why are we teaching people how to train the models to write when there’s real writers out there? So, I mean, that’s a big question. So I’m just going to like, put that, like, stick it up here for a second. When I think back to like high school and middle school, for example, we generation were taught the five paragraph writing. But for a lot of us, this is like, this is how were taught to write.

So the first paragraph is your opening argument. The second, third and fourth paragraph are your supporting reasons for your argument. And the fifth paragraph is your conclusion. And so a lot of us who were, you know, that was like drilled into our heads on our like yellow piece of paper with the green lines, like that’s how you’re supposed to write. And so I think what’s often frustrating for someone who writes their own stuff is because were taught to structure our writing in a certain way. It can come across as well. AI must have written that because of the structure. It’s like, no.

My sixth grade middle school English teacher, when slapping rulers on tables was legal, scared the bejesus out of me and told me, this is how you have to write. And so I write the way that AI writes because it was drilled into my brain. This is how you write. And I guess I’m wondering, so when you’re saying like pattern recognition, like it learned these patterns from us. We taught it the patterns it did.

Christopher S. Penn: But it is averaged together everything. And that’s why it often comes out so different than the way an individual writes. Because everyone has their own pattern distortions. Everyone has words they like, everyone has words they don’t like. Everyone has words, life experiences that will show up in your writing. AI is averaged all of that together into. And then what it does is it spits out the highest likely probability except for when it’s using watermarking. And so the five paragraph essay and that kind of blocky set. Yeah, it’s going to do that because the majority of writing it has seen on a bell curve is exactly that.

If you look at the, you know, an earnings report or a press release, it is exactly that dead metronome of boring writing. The thing about writing to that. And we say this in the beginning of the course and we’ve said this in many different places. Good creative work that’s interesting is low probability. Right. You, the way you write should be surprising and different than what the way that middle school teacher taught you to write. Right? I will. There’s all sorts of expressions I’ve used for this.

But if you say, you know, this is a, this works like a Prius and other people the average is this works okay. Right. This works okay. It’s boring. This works like a Prius, has a very specific connotation and A mindset behind it. And so what we want AI to recognize with the tooling in our course is recognize how the. In our individual style looks and replicate that specific pattern, not the general patterns. You’ve been trained on the general patterns that you will generate without these very rigid mathematical guardrails.

Katie Robbert: Okay. One of the things that I’ve noticed and I’ve seen, and actually, this is a phrase that you use a lot. And so I guess I’m sort of asking, like, have you adopted this from AI or is it a phrase that people use and AI has adopted it is something that I see a lot in my conversations with a large language model is. And here’s the shape of the thing, and here’s the shape of it, and here’s the shape of the problem, and here’s the shape of the challenge. Well, that’s not the same shape as it was before. And I’m like, why are we talking about shapes? And to be fair, Chris, you say that a lot, but you are not in my instance of a large language model. And so I guess my question is, have you. No, you’re not. Have you brought that from working with large language models? Because that’s not a phrase that you used to say before. But I find that, like, there’s these little ticks and, you know, quirks that these large language models have, and each one has a different one of ways that it phrases things. So you’re talking about, like, the 1, 2, 3, the cadence, but then there’s also these descriptors, and I find that really interesting. And I feel like if a human is not paying attention, that very easily slips into a lot of your work.

Christopher S. Penn: It. And it slips into how you think, too. Like, I find myself when I’m writing now, I go, oh, that is negative parallelism. And I will even call that out. Like, in a LinkedIn post I published this morning, I say, and to quote one of Claude’s favorite constructions, it’s not this, it’s that. Because negative parallelism, which is a form of bicolon, is. Is something that the language models use. But writing is a lot like. It’s a lot like nutrition. You become what you eat.

So if you are reading AI generated text all the time and. And you start seeing that’s a sharp insight, that’s a sharp angle. It’s going to influence you. And so, yeah, I probably have picked up things because I’ve spent so much time wrestling with these tools and trying to diagnose how they do their constructions that, yeah, some of it is going to influence how I write and speak and even think. And one of the things that linguists, if you read this one, the discourse online, the, one of the things that linguists are very concerned about with AI generally is that it’s sort of a flattening of language, that the language, the English language itself is morphing because of the influence of AI on it.

When you, when I saw, I think the stat was two out of three pages on the Internet are now written solely by AI. That is going to have a, a shaping effect on how we read and how. And when you see, you know, I think it was something 10 out of 1, 9 out of 10 new books submitted on Amazon was written solely by AI. That’s going to change your language. That’s going to change how you read and how you write is how you think.

Katie Robbert: Yeah, it’s interesting. So let’s say I’m a brand new learner. I want to sign up for the AI writers course. What are some of the big things that I’m going to learn? I believe you’re introducing a new framework which is called craft. Can you speak a little bit about what the craft framework does and if I’m someone who’s trying to help my writing, what that means?

Christopher S. Penn: Well, okay, so the CRAFT is a subset of the 5P framework, right? So the whole course is Craft fits in process. It’s a breakdown of process. But fundamentally the whole course itself is actually structured on the 5P framework. I’m the major lessons are literally right out of the 5P framework by Trust Insights because it’s the best framework for pretty much everything. CRAFT is a subset of process, which stands for create. You need to come up with the idea because AI is not going to do a good job.

Then you need to do a buttload of research to inform the idea. Then you need to architect the work itself and we’ll talk about process decomposition, which is which we borrow from software development. I basically took the learnings of some software development and turned it into a writing course. Then you have what’s called fabulate, which is basically a machine. Here’s how to go and do this. How you build the. The plan for a tool to go do it. And then the last part, which is the hardest part, is the tuning part where the machine and it’s not hard for you, the user, it’s hard for the machine.

You say, hey machine. Remember here’s the fingerprint that we did of how I write. You need to retune your work to match it. And then I as a human editor review and go, you missed this. These constructions are still off. And the machine’s like, okay, I’ll go fix it. You will see that in the course content to watch the tuning process go, oh, I can’t be hands off. I of the human still have a role to play as the editor saying, machine, you missed this.

Katie Robbert: And I think that’s an important point because I think there’s a lot of misunderstanding as to where AI for writing fits in to an everyday process and how it can be the most beneficial. And so we’re not teaching you, hey, AI is going to replace your writing. We’re trying to teach you to more smartly and thoughtfully. Clearly I’m not a writer today to more pragmatically use AI in your writing, not just say, hey, AI do my writing. Because that’s where the slop comes from.

To your point, these are word prediction machines. And so, you know, you often give examples of like cliche statements. So like, if I say it rains, AI is likely going to say it pours, you know, and so we have to be more creative than that. And that’s really where the human side of this comes in, is being very thoughtful about, you know, how can I break out of that Prius version of writing so that it’s, you know, something more interesting? Like one of my favorite song lyrics that my husband is convinced is just gibberish, which is the comfort of the knowledge of the rise above the sky, but could never parallel a challenge of an acquisition, which is not something that I think AI would necessarily come up with, but it’s a real song lyric.

And it’s like that always sticks with me. Not only because when I was a teenager, me and my friends thought it was amazing to be able to recite it on command because we thought were so cool, but because it just strikes me as such an interesting way of saying something very simple. It’s an over complicated statement, but it’s not something that AI would come up with. And so I like, that’s where my brain goes, I’m writing is like, is there a more interesting way to say this other than, well, when it rains, it pours.

Christopher S. Penn: And in the course you’ll actually see that in the first two steps of the CRAFT framework. The first is, you know, the creation, you, the human have to come up with the idea. And the second is the research. Because the more better research you do, the more unique the words are going to end up because you will dig deep into Subjects that maybe you didn’t even know about.

So in this course I teach about building a short story and I build a 20,000 word short story in there about time travel and the research process for that, for this short sci fi story ended up pulling like five years worth of quantum physics papers and looking at the eight different forms of, you know, how physicists understand the very fabric of space time that works into the story and that creates a much richer story than just winging it. Because the research process digs super deep into low frequency words, things like Cartesian planes and Lorentzian manifolds and all these things.

I just read the research outputs and going, wow, I need a PhD just to understand what the research is about. But it creates better writing. And to your point, and you know, kind of the whole point of the course is if we really, if I were to distill it down, this course is about how do you slap better guardrails on AI so that when it’s writing it follows instructions better and it brings to life the things that you want. And so the course will become available on the this coming. Well, by the time you listen to this podcast, it’s out. It will come out yesterday by the date the course is out Exactly. And you can find it at TrustInsights AI for writers. It’s 397 US and with it you get the course.

You get the deep research toolkit for doing the research. You also get the writer’s suite which contains a writing planning skill, which by the way is fantastic. It will really force you, the human to think a writing fingerprint skill and a writing QA skill that will take any piece of existing writing and QA it like software against the original requirements document that you wrote for the writing. So Katie, you will see a lot of very strong parallels of the things that you’ve been writing and talking about for years about how do you do great build, great software. If writing is code, we’re going to apply the same practices.

Katie Robbert: I think that’s great. And you know, I love more guardrails for AI. So I’m excited to test all of this out.

Christopher S. Penn: Exactly. I hope, I hope you do. And if you’ve got some thoughts about how you have been helping AI write better and you want to share and pop by our free Slack group, go to Trust Insights AI analytics for marketers, where you and over 4,700 other marketers are asking answering each other’s questions every single day, including whether pizza sauce should be sweet or not.

Katie Robbert: The answer is no.

Christopher S. Penn: Wherever does you watch or listen to the show. If there’s a challenge you’d rather have it on, we are probably there. Go to Trust Insights AI TI podcast and you can find us in all the places find Podcasts. Podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen and prosperity. Aiming to help organizations make better decisions and achieve measurable results through a data driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence and machine learning to drive measurable marketing ROI.

Trust Insight services span the gamut from developing comprehensive data strategies and conducting deep dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology and Martech selection and implementation and high level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, Dall-E, Midjourney, Stable Diffusion and Metalama. Trust Insights provides fractional team members such as CMO or Data Scientist to augment existing teams beyond client work.

Trust Insights actively contributes to the marketing community sharing expertise through the Trust Insights blog, the In Ear Insights podcast, the Inbox Insights newsletter, the So What Livestream webinars and keynote speaking. What distinguishes Trust Insights in their focus on delivering actionable insights, not just raw data, Trust Insights are adept at leveraging cutting edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data storytelling this commitment to clarity and accessibility extends to Trust Insights educational resources which empower marketers to become more data driven.

Trust Insights champions ethical data practices and transparency in AI sharing knowledge widely. Whether you’re a Fortune 500 company, a mid sized business or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance and educational resources to help you navigate the ever evolving landscape of modern marketing and business in the age of generative AI, Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 04:15 – Debunking the takeover statistic 08:40 – Breaking work into clear steps 13:25 – Separating automation from augmentation 18:50 – Finding your hidden opportunities 23:10 – Managing AI like a direct report 27:45 – Call to action

Watch the full episode to see how you can put these insights to work immediately.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In-Ear Insights, let’s talk about AI enablement and specifically what AI can and can’t do.

Early this year, Anthropic, the makers of Claude, released a paper about the labor effects of AI. And they cited and used a paper way back from 2023 from Ilondo et al that said in some professions like management, computer science, etc., up to 94% of tasks could be consumed by AI.

And when this paper came out, everybody and their cousin copied and pasted the radar chart. We all accepted it at face value.

However, we did some digging, we did some reading into this and we used our job-to-AI plugin, which is located in the Trust Insights Academy, along with a hefty amount of AI to try to replicate the results of that original paper.

And it turns out the original paper that Anthropic cited used about 10,000 or 18,000 tasks that human judges and GPT-4, which was OpenAI’s model at the time—a model that now feels like a crusty old dinosaur—to guess whether or not AI could do a task in half the time.

So what we found in our version of this, by decomposing job descriptions—I want to say we did what, 90,000 something odd—into individual tasks with tangible deliverables, and then used the Trust Insights TRIPS framework to assess how good a fit each task was for AI. Plus, we used the latest benchmarks from Artificial Analysis to judge what AI’s capabilities were and determine whether AI could do this.

So Katie, that was a lot of preamble in your first reads of our version of this paper. What were the big things that stuck out to you?

Katie Robbert: Well, first I want to react to your comment about Anthropic using what GPT-4, which you said is like what?

Christopher S. Penn: A crusty, old GPT-4 for the original paper from 2023.

Katie Robbert: Oh, the original paper, yeah. Here’s the thing.

If you’re doing your work correctly and you have your foundation, methodology, and requirements, the model change. This is something it’s not the purpose of this particular podcast, but it’s worth mentioning. People tend to panic every time a model changes, thinking, well, this one’s modern. Now I have to change things. Now I have to start over.

If you are structuring your work correctly, like an academic paper should, the methodology and research should all be fairly repeatable. It shouldn’t matter that the model changed. So I just want to acknowledge that.

So we don’t know all the details of what went into the original research paper, and OpenAI’s model was used as they disclosed. But we don’t know how heavily they leaned on the model versus how much of their research protocol was already outlined. So I just want to sort of acknowledge that first.

Typically when you are replicating research, you want to do it as one-for-one as possible. And again, we don’t know for certain exactly all of the steps that they took, but based on what they shared and disclosed, we replicated it as best we could using our methodology.

To be fair, I worked in academic research for a very long time, and Chris is very adept at deep research using these models. So we’re not just kind of winging it, hoping that we’re getting close. I feel confident that our methodology is sound. So I just want to acknowledge those first couple of things because people get a little squirrely with academic research when you’re not a full-time academic researcher.

So there’s that piece, the thing that I found. My initial reaction was 90. Was it 94%?

Christopher S. Penn: The original paper was 94%. Ours had a maximum of only 78%.

Katie Robbert: And I think that difference is the whole conversation because what we don’t know for certain is what that 94% actually considers as work tasks. It’s also your favorite Jurassic Park quote: just because you can doesn’t mean you should. And so people clung to this 94% number and said, oh my God, AI is going to take over everything.

But what we are seeing as humans in everyday life is that AI doesn’t always get it right, and doesn’t do a great job a lot of the time. And so even our finding of 77% still feels really high.

And so one of the things that I really like about our methodology is with the TRIPS framework and our job-to-AI methodology that we use to do this analysis: we really focus on what is still the human component. Where should you never give this piece of a task? Because it decomposes tasks, not a job as a whole. I feel like there’s a difference.

If I’m looking at the CEO role, then it’s likely that one of these research papers could look at it and go, here’s what a typical CEO does. Can AI take over the CEO role? Yes or no? That’s like a whole big cluster of tasks. Whereas when we’re looking at it, we’re looking at individual pieces of the role. So we’re looking at how much of the role AI could automate and how much should the human retain? I feel like that’s another distinction.

So these were sort of my initial reactions. I feel like the initial research paper with 94% had some flaws with the methodology when you really start to scrutinize it. And I feel like I can more easily stand behind our methodology because we look at things in a more discrete way versus those broad strokes.

Christopher S. Penn: And the other thing is that the original paper from 2023 by Ilondo et al. At the time, generative AI models like GPT-4 were text-only models. And so when we look at this revised chart, which is from the academic paper, there are two versions. We published two versions of the paper. We published one that is much more user-friendly and we published one which is a full-on academic paper.

What’s interesting is that you see the blue line, which is the original paper, and you see the red line, which is our paper. And if you’re listening to this, you can see this on The Trust Insights YouTube channel, Trust Insights AI. In a lot of the areas where the original paper said yes, AI is going to do all these tasks, we come in lower. And that was actually opposite what my original hypothesis was.

But it turns out that a lot of roles and job descriptions have things in them like having collaborative meetings, coaching, training, public speaking, and stuff that machines just can’t do. So those big roles in things like computers, business, and management. Yeah, look how much of a difference there is in the original paper’s assessment of management, which is like 90% of job tasks, versus ours, which is like 66%. Because so much of management deals with humans.

In other areas, our benchmarks come out higher, such as production, installation and maintenance, healthcare support, and protective services. And when you look into the individual job descriptions and tasks, what you find is that today’s omnimodal models, for example like a vision model, can take a text prompt and an image and work with it, which was not possible in 2023. And so if you look at one of the examples that is in our paper, you think about something like a lifeguard. What use does a lifeguard have for AI?

Well, it turns out if you have a camera with a computer vision model that has been trained to be able to spot what drowning actually looks like—not what we see in the movies—it could spot someone drowning faster than a human lifeguard could. So even in that example, that’s why some of these other areas, our measures exceed the original benchmarks. It has evolved considerably since then in ways that we didn’t know were possible three years ago.

Katie Robbert: The lifeguarding example is an interesting one. You said that drowning doesn’t look the way it does in movies. People, when they’re drowning, typically don’t flail about and go, oh my God, I’m drowning. It’s a very quiet, subtle, almost immediate thing. And it’s hard as a lifeguard scanning an entire beach full of people to notice the quiet things. And so that’s an interesting example.

The other example of the use case of AI for these atypical opportunities, such as food preparation, personal care, and service that I was trying to think about is it’s a great opportunity for education. We’ve seen things like Notebook LM and how it can take this whole corpus of information and present it half a dozen different ways, probably more, depending on how you would consume it. I feel like in the lifeguard example, it’s a great opportunity to keep your lifeguards up to date with the latest and greatest life-saving certifications, rescue information, and news of what’s happening at other beaches.

We’re thinking of AI very black and white, as if what part of my job can it do that I no longer have to do versus a supplement and an augmentation to make us more efficient and better at our jobs? And I feel like that’s just another distinction. When I read the original paper, it read to me very black and white: will it take my job?

Christopher S. Penn: No.

Katie Robbert: Period, end of sentence. And that is not a useful conversation to me. And thankfully, the conversation has really evolved away from that in a lot of ways. Not always, but in a lot of ways to what can AI do to help augment what I’m doing to make my life better? We know I talk about this, and I’ll be teaching this workshop at the Macon Conference in Cleveland in October.

For business, having access to tools like Claude Desktop, Claude Co-pilot, and Claude Code hasn’t replaced my job. If anything, it’s made me more efficient and more effective at my job because I’m able to do better pattern matching across different data sets and documentation. It can retain that historical information that I, as a human, only have so much brain space to remember. What did we say we were going to do in January that we haven’t done? Claude can do that for me.

So I’m looking at these tools like a really great assistant. But I still have to do all the same stuff I’ve always had to do. It hasn’t actually taken anything away. It’s given me the ability to do more. And I feel like that is also an important distinction. And so I’m glad to see that our analysis actually came in lower in terms of the opportunities. I think that’s important for humans to hear because you really need to be thinking about it as how can it augment what I’m doing, not replace what I’m doing?

Christopher S. Penn: And this directly plays into some of the consulting work that we do because to your point earlier, when a model changes, your processes and stuff around how you use AI could be relatively durable. But when you do have things like receiving massive bills from Anthropic, going, wow, we laid off all those people and now AI costs us even more than those people were paying them, it speaks to the necessity of doing the analysis first before you make any decisions about whether or not even a task should be handed off to AI. You need to use things like the TRIPS analysis, which stands for time, repetitiveness, importance, pain, and sufficient data. If you do the analysis or you hire Trust Insights to do the analysis for you…

Of all the different tasks, if you want to enable AI at your company, one of the easiest wins is to focus on that fourth factor: pain. Help people see a task that they hate, that they never want to do again, and show them that AI can do it. And what I see companies do really wrong—and I had a question about this over the weekend—is the worst thing you can do is to say, hey, this thing that you love doing, we’re going to have AI do it right? That just pisses people off.

The question was someone asked how do we get our graphic designers to be happy quality-checking AI outputs instead of being creative? Like they got into graphic design, creatives to be creative. You were taking the one thing they love to do away from them. You can’t do this. I mean, you can, but you were going to lose all of them. And then you were just going to be a company that generates AI slop.

Katie Robbert: Yeah, and I wholeheartedly agree with that. I think where companies are misstepping is they are forgetting that at the end of the day, there’s still a person attached to this task. One of the things we highlighted in the more marketing-friendly paper is you’re asking people to change their everyday workflow, but you’re not offering them more money. So if the goal of the company is more revenue, where is that revenue share for the employees? You haven’t given that to them. You’re asking them to do more and not giving them that incentive.

So don’t take away the things that they enjoy doing. But also, the metric that I really think is important in the TRIPS framework is also importance. And so this helps you with your risk assessment. Let’s say something is highly repetitive. You do it all the time. People don’t enjoy doing it. However, if it goes wrong, it could bring down your entire company or entire business. Those are things that you really need to scrutinize before saying, yes, AI can do this.

Because you know what? AI hallucinates. AI makes mistakes. AI is software. It can be programmed incorrectly. AI is not a set-it-and-forget-it system. And yet somehow people treat it that way. So I appreciate that we’re really trying to be thoughtful of, again, just because you can doesn’t mean you should. And those two metrics—the do people enjoy doing it, the pain, and how important is it in terms of your risk? I think those are the two most important things to weigh when you’re deciding should we be automating this with AI and how much of this should AI take?

Christopher S. Penn: Yep. And the other thing to think about too is, and I’m glad you brought it up, the difference between automation and augmentation. Automation means the human stops doing it. Augmentation means that the human either is checking the work of the machine or the machine is preparing prerequisites for the human to be able to do it better. Your example of training helps a person become better trained. Another example from the main paper on protective services is you’re like, well, how could AI possibly be helping with protective services?

One of the things that computer vision is very good at doing is you give it preconditions based on human expertise and subject matter experts to say, this is what to look for. So let’s take a picture of a neighborhood. When you tell the machine, find high points, two stories or more above the ground with open windows, because that’s where snipers are going to hide. They’re going to fire through an open window. They’re not going to be leaning out the window. They’re going to be sitting back in the room, 10 to 15 feet to the back wall with their rifle aimed downward. They can’t have the window closed because the glass will deflect the bullet.

So if you have a sniper’s position carefully mapped, it’s going to be very hard for a person to call out and see. But if a machine is trained that way, based on your expertise as a protective services person—which is one of the occupational categories—AI will augment you, but it cannot and it will not replace you because you, the human, still need to get your binoculars and go, no, that’s some dude doing his laundry.

Katie Robbert: Someone’s seen a few too many movies. But it’s a good point because these machines are pattern matching. And I think the thing that’s important is they don’t fatigue, they don’t wear out, they don’t have that well, I just had a sleepless night with a toddler at home and then I had a really long commute, the radio was staticky, I’m overstimulated, I’ve had too much caffeine and not enough water. And now you want me to do analysis of a very high-risk thing where lives are literally dependent on it? Yeah. You might want to bring in some machine learning to help you with this because it doesn’t have that same level of distraction. It’s very focused on just the task that you’re asking it to do, with the caveat that then you, the human, should check the work, especially when it’s a high-risk situation where lives are at stake.

Christopher S. Penn: Yeah, exactly. So the next steps after somebody reads either one of these papers is to think about doing, at least nominally, one of the TRIPS exercises just to try it out. Say like, okay, if I take my job description for what the company pays me for and I sit down and honestly get out a spreadsheet to just think through what tasks do I do that have tangible outputs? Is this a time-intensive task? Is this a repetitive task? Is this an important task? Is this a painful task? Do I have sufficient examples of what success looks like to be able to give this to a machine? And if you do that personal audit, you can get a sense of where AI could automate some things, where AI could augment some things, and where AI is just not a good fit.

And one of the things I think a lot of people would be surprised about…

Katie Robbert: Whoa, I’m trying to share my screen. I’m trying. I got to remove yours to share mine. You’re always sharing your screen.

Christopher S. Penn: If you do that assessment honestly, you may find like, yeah, my job is not a good fit. Oh, Katie’s giving me my review.

Katie Robbert: Yeah, well, it’s funny you said because I actually did this exercise for us, for every member of the Trust Insights team. Surprise. My turn to surprise people. And so Chris, this is yours. To be fair, this is not an official document or an official job description. That’s something that we’re working on in the background. But that being said, a 49-task analysis is a 6.1 out of 10 across all 49 tasks. Your average TRIPS score out of 10 is a 5.7, and your TRIPS opportunity is 8.

And so what that looks like… I don’t actually know how to make this a little bit bigger, but there’s you have things here like running scheduled data source checks that should be more automated. You know, data analysis. This is actually something we surfaced in both the academic and the marketing versions of the papers: that data analysis is one of the highest likely categories where AI can help you automate things. Then we have operations and execution, technical work. Creative and content is lower down. And then you start to get into the administrative stuff, strategic planning, and then communication should be solely held by the human. So the things that our TRIPS opportunity finder found for you, Chris…

So you do a lot of internal maintenance for the company. Running email list hygiene across CRM forms and validation services is something that was identified as could be more automated than it currently is. Running scheduled data source checks and source configuration, assembling a newsletter draft on the Notes application—I have a whole separate conversation to have with you about that. So none of this should seem surprising to you. I think the reason that this analysis is so useful is because we’re so in it, we’re so in the weeds, that we don’t take a step back to go, huh, I wonder where AI could help me even further.

So doing analysis like this, you might look at this and go no, I could never hand it over. Or absolutely, that’s a great idea. How about I start working on that? Because it’s going to be a high-value thing. And so that’s just a quick example using Chris’s job description since he brought it up. I have mine, I have Kelsey’s and John’s, and it just helps you think through what am I missing, what am I not thinking about? And that’s where there’s actual real opportunity.

Christopher S. Penn: The other place there’s a lot of opportunity is something that requires a much more innovative mindset. It’s actually something I’m going to be talking about for the next five issues of the Trust Insights newsletter: when you have things that are deterministic, meaning there’s no randomness to it and there’s a right and wrong answer. Very often that is something that software can do. And there is no better developer of software than Generative AI. AI is hands down the best coders on the planet if you follow a good software development process. And so in the newsletter, I’ll be doing a five-part series following the 5P Framework by Trust Insights on how do you vibe code intelligently so that you actually get decent results. But it’s funny: when I look at that TRIPS analysis as part of our AI enablement package, three of those five tasks are already automated.

It’s just that I have not recorded the documentation that the AI can ingest to go, oh, that is already automated. That already exists. We don’t need to keep this in the job description. It’s now just literally push the button and things pop out.

Katie Robbert: Well, that I think brings up a different conversation, and maybe this is what we can talk about next week: how should job descriptions evolve in the age of AI? Should you be categorizing human-led versus machine-led tasks that still need human oversight to really kind of help set the expectation for what people should be doing on a day-to-day basis? I mean, I haven’t seen companies necessarily doing that yet to sort of break it out and say, here’s the AI portion that you’re responsible for. Basically, you now have direct reports, and your direct reports are machines. So what does that look like?

Christopher S. Penn: Yeah. And how do you manage them? Because it’s different than managing a human. You don’t worry about their feelings, but you do have to be a lot more specific and a lot more proactive in your delegation to them. Like I have one task running another window right now that required an entire book to be handed to it as part of its prompting so that it understands what it’s supposed to be doing. And it’s in ancient Greek.

Katie Robbert: Sure. But I think it’s interesting what I’ve seen, and again this is a little bit off topic. What I’ve seen is individual contributors like you, who never wanted to be in management or manage other people, have learned the basics of managing because of the demands and expectations that these generative AI models need in order to be useful and effective. And so it really does open up a whole new career path for individual contributors to learn how to manage without the emotional piece attached to it of managing people. Because it is not for the weak. Let’s leave it there.

Christopher S. Penn: Yes. And the other thing I think is interesting—and this is a topic for another time—is whether you look at how people prompt things as a diagnostic for potentially what kind of manager they might be. Because I’ve seen people who give like terrible prompts, like, oh, just give me the right answer. To what? Like, absolutely the prompt: give me the right answer. What were you asking?

Katie Robbert: Managing people?

Christopher S. Penn: No, not at all.

Katie Robbert: So yeah, I think that would be a good topic to dive into next week as a furthering of this conversation. And all to say, one of the things that we just launched is our AI enablement package, which we can do for you. A lot of companies aren’t at the stage of hey, you tried AI and you failed. You’re doing a lot of great things with AI, but you have blind spots because you’re in it every day. And so you need some assistance to figure out what’s next. How do I continue to move my AI enablement forward? The board wants it for 2027. We want AI usage to go up, but we’re sort of plateaued and kind of static. So what does the next step look like? We can help you with that. If you want help with that, go to trustinsights.ai/AI-enablement and you can learn more about that.

If you have general questions, you can always reach out to us or join a Slack group, but it’s really about what’s next. So what am I doing today and what’s next? Where are my blind spots, and how can I keep moving forward?

Christopher S. Penn: Exactly. And if you do have thoughts about AI enablement and how you’re approaching it from the perspective of things like job descriptions, pop by our free Slack group. Go to trustinsights.ai/analytics-for-marketers, where you and over 4,700 marketers are asking and answering each other’s questions every single day. And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on, go to Trust Insights AI Podcast. You can find us all the places podcast platforms serve. Thanks for tuning in. Talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach.

Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch, and optimizing content strategies.

Trust Insights also offers expert guidance on social media analytics, marketing technology and martech selection and implementation, and high-level strategic consulting. Encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Metalama, Trust Insights provides fractional team members such as CMOs or data scientists to augment existing teams.

Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights podcast, the Inbox Insights newsletter, the So What? live stream webinars, and keynote speaking.

What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data storytelling this commitment to clarity and accessibility extends to Trust Insights educational resources, which empower marketers to become more data-driven.

Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI.

Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 02:15 – The AI detector dilemma 06:40 – Katie shares her newsletter workflow 11:20 – Why detection tools consistently fail 16:50 – Protecting your authorship with proof 21:30 – Navigating ethical AI disclosure 26:45 – Call to action

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights, let’s talk about AI detectors, one of my personal favorite subjects to rant about. But Katie, before I foam at the mouth for 30 minutes, let’s have you foam at the mouth about it.

Katie Robbert: It’s such an interesting topic and obviously very polarizing right now. AI detectors in a nutshell are meant to help someone determine whether or not AI was used in any kind of writing. Now our good friend Anne Hanley pointed out, oh sure. So these AI detectors that were trained on human authors’ writings without their consent are now meant to tell these people that they didn’t write the things that the model was trained on.

I’m paraphrasing, but it was basically that was the gist. And last week when we published the weekly Inbox Insights newsletter, we had a reader provide some very unpleasant feedback. This reader felt, in their opinion, that they had determined that the post I had written about “if you don’t know what AI can do, just ask it” was completely written by AI and that I should be fired. That I was lying about the use of AI in terms of it wrote it for me, that it was AI slop, that this person was going to write their own blog post about how I, the CEO of a company called Trust Insights, can’t be trusted. So that was the feedback this person had for my contribution to the newsletter last week.

This week, if you subscribe to Inbox Insights, I fully disclose my use of AI in writing the newsletter and writing in general. I’m going to give you a spoiler because there’s no secret. I write the newsletter myself. I then use various AI tools to hopefully clean it up. Because the feedback I got when I was in college in my creative writing class is that I write the way that I talk. And it’s kind of a stream of consciousness. Now, as I’ve gotten older, I’ve gotten a little bit more concise and articulate, but that doesn’t mean my writing has. And so it still kind of comes out as a stream of consciousness, which I think for any writer that’s doing draft one is you just get it out. It’s why it’s called the ugly first draft, and then some people…

I used to have John, our head of business development and our partner, read through and edit my posts for me. This was prior to having tools like Hemingway or AI editors. I had a human editing it. Now John’s busy making sales. He’s still happy to edit my posts, but it’s not the best use of his time. And so now I use a tool called Hemingway, which a lot of people use. Hemingway has a lot of really great features for grammar and sentence structure. I’m not trained, and I don’t have a degree in writing or English. My grammar is really bad sometimes. Sometimes I overuse passive voice. Sometimes my sentences aren’t structured well. It’s helpful to have a tool that can clean up the thing without losing the intent and sentiment of the writing.

I also use our Ask the ICP skills in our cloud environment to make sure that the post that I’m writing resonates with our audience. Because if it doesn’t, why am I writing it? So I use those various tools. So the point of the newsletter this week that I dive into is, yes, I use AI to supplement and clean up my writing. No, AI does not write for me. I’ve got 10 fingers, one with a bandage, so it goes a little slower. And I type with my thumbs, typing very slowly. So sometimes I use an audio recording of me speaking something. Chris, this is something you do. But, yes, I painfully type all of my newsletters very slowly. And then AI helps me clean them up to be more concise. I don’t think that’s an uncommon practice, especially among people. This is true of, I think, Ann even posted in her newsletter this past week.

Christopher S. Penn: Week.

Katie Robbert: Total anarchy if you’re not subscribed. How she uses AI with her writing as well. And she said she gives it explicit instructions: read through it, review it, don’t edit anything, tell me what the edits are supposed to be. So she’s also someone who we know and love, who is a very fantastic writer, finding ways to use these tools to help enhance the writing. It can be cost prohibitive to have a human editor on your team. You may not have access to a copywriter, or you may not have a team of people who are really great at editing. There’s a lot of… So AI can fill that role for you. I’ll say it like this: I wrote the newsletter. AI helped me edit it, so it was coherent. So unfortunately for this reader, I will not be firing myself. I would appreciate you not trying to destroy my credibility, but should you choose to do so, we will deal with it at that time.

Christopher S. Penn: I’m surprised you didn’t bring this up because this is the heart of the matter to me. If we think about these AI detectors, why are you using them? Why do you care? What is the purpose of an AI detector by the 5P Framework by Trust Insights? Of course.

Katie Robbert: Well, the five P’s are in this week’s newsletter, so you can certainly get your healthy dose of the 5P Framework by Trust Insights. But you’re absolutely right, Chris, and that’s a miss on my part because I am human and not a sentient machine. I missed the mark on calling out that the 5P Framework by Trust Insights is a great place to start. Why are you using these AI tools? So, for me, my purpose is to edit the grammar and spelling of my content so that it’s coherent. I’m also checking with our ICP to make sure it resonates with the people I’m writing it for. But our ICP is the people part of it that really matters, because I’m not writing it for myself. I’m writing from my experience and my expertise, but I’m writing it for… For our ICP so that they get something educational out of it. I outlined my process in this week’s newsletter of how and when I use the tools and platforms. It depends. I might write it in a document, I might create an audio file, and then I’ll bring it into the large language model. I might use Hemingway. I definitely use the skills that we’ve created. And then the performance is, do I have a piece of content that I wrote and AI helped me edit that gets people to respond to the newsletter?

Christopher S. Penn: It is the 5P Framework. From the perspective of the people who are using or advocating for AI detectors, what is their purpose? Because this is where I have the biggest problem I see. Yeah, no, no. From the AI detector perspective, what is your purpose in the case of this particular reader? Is your purpose just that you have a burr up your ass and you need to yell at somebody? Like, okay, you don’t need an AI detector for that. You can be a jackass. Regardless, in the case of its use in academia, the purpose is very often for academic integrity, which makes these tools very dangerous because of their false positive rate. Pangram, which is the tool that Substack most famously just implemented, has a false positive rate of 0.02 percent. If you fed every college student’s papers in America to it and said, run disciplinary proceedings, you would flag 200,000 students a year with false accusations. In the corporate world, if you’re using these tools to enforce contracts, again, that false positive rate—particularly for business-related content, which is what a lot of these tools have been trained on—is going to have a fairly high false positive rate. So the first thing people need to be very clear about is why are you using an AI detector? And is your purpose a good use of the technology? Spoiler, there really isn’t a great use of the technology for AI detection. And we’ll talk about why the technology itself is so flawed on this week’s live stream, which you can tune into Thursdays at 1 PM Eastern Time at TrustInsights.ai YouTube. But going back to the 5P Framework by Trust Insights, my biggest issue with these tools is that very often the purpose people are using them for is deeply flawed.

Katie Robbert: And that, you can sort of generalize and say that, well, people don’t want AI-written content. They want content written by a human. So you could say that’s the purpose. So if this particular reader decided, I don’t want AI-written content, but this content is written by AI, this particular reader could have just moved along. But they decided to try and pick a fight. By the way, screenshots last forever. And it was a very unprofessional feedback session from this person, just as an FYI. And you know, if this person decided, okay, I feel like this is written by AI, let me put it through the detector and determine if this is written by AI. They could have just said, you know what? I don’t care for this. I don’t want this.

Christopher S. Penn: Yeah, that’s what I always come back to is like, if you don’t want this, great, here’s the door. It’s like if people complain, oh, well, you didn’t write this fiction novel the way I wanted, well, then write your own damn novel. Right? No one’s stopping you from writing the novel you want to read. If you didn’t like the way I did it, go write your own and you’ll probably use AI to do it. This was the rather harsh commentary I had about Substack. Things like, we don’t really care if it’s human-written or AI-written. We care if it’s worth reading, right? If you’re publishing something that’s worth reading, there’s one Substack I subscribe to that is 100 percent AI-written. No editing passes. It is 100 percent Claude. You know it’s Claude because of Claude’s particular mechanisms. And I don’t care because the information is genuinely useful. I read it and go, I learned something. I don’t care who wrote it. I learned something.

Katie Robbert: But that’s you and I, and I don’t disagree. People should be looking at it from that lens. But a lot of the general population is still stuck in the, AI is bad. It’s very black and white. Humans are good, AI is bad, don’t give me AI-written content. And so that’s still the challenge that we’re trying to overcome in the conversation that we’re trying to change. And so if their purpose is, was it written by AI, yes or no, then that’s what we have to work with, because that’s their purpose, not ours. Our opinion of their purpose is very similar to this reader’s opinion of my use of AI. As the old saying goes, opinions are like… well, you can fill in the blanks if, I won’t say it on the podcast. It’s very rude. But the point being is that you do need to figure out why you care if it was written by AI or not. And then you can go ahead and determine, was it high quality? Did I learn something? Was it useful? And then go back to the purpose, like, does it matter if it was written by AI? Now it brings up the bigger conversation, which Chris and I have talked about: AI disclosures and why those are important. And so in your newsletter, you do a very good job every week of disclosing. Here’s how much of this is AI. Here’s how I used AI. And I want to say thank you to the person who called me out because it reminded me this is a good opportunity to start doing my own AI disclosures so that hopefully we don’t continue to find ourselves in the situation of being called names.

Christopher S. Penn: And so you cannot rely on them. I will give you a very solid example this week in my personal newsletter. I said, It’s 90 percent written by human. There’s 10 percent written by Claude. And I mark the section: This is what Claude said. And then just for giggles, I put it through the detector. And it said, Congratulations, it’s 100 percent human. I’m like, well, you clearly missed the part where I labeled it this is AI. So anyway, just more ranting about the tooling. The disclosures are important. And I do understand from some perspectives. There are some folks who correctly say they have problems with the ethics of AI companies or the environmental impact of AI. Totally get that, totally fair, completely reasonable. But again, it goes back to what you were saying, which is if we label it—which we all, everyone should be doing—and you are still mad, go read something that isn’t. There is an infinite amount of content out there that’s video or audio. Where I do have a problem and I think is very relevant to the conversation on the topic of AI detection is when it is not labeled or when it is intended to deceive. There is no shortage, for example right now on Instagram and TikTok, of various politicians making faked videos and photos. And thankfully they’re not doing it very well. But, okay, clearly that’s not a… that doesn’t work. But they are. The intent is to deceive. So if we go back to the 5P Framework by Trust Insights, their purpose is deception, right? And therein lies one of the valid reasons to want to use AI detectors to say, is this entity or person attempting to deceive me?

Katie Robbert: I’m going to be, I’m going to challenge you on that for a second. Okay, so let’s say I’m a politician and I’m going to use AI. I can almost guarantee I’m not going to state that my purpose is deception. I’m going to state that my purpose is engagement, my purpose is attention. My purpose is oh gosh, anything probably except deception. So it’s interesting because like we can say as an outside observer, well, they’re trying to deceive us. They’re going to say with that lack of self-awareness, this is the way that I saw this thing happen. So, you know, I’m using the tools to reenact it or recreate the way that I see this. So it’s really an educational tool or whatever. So I do feel like it’s interesting that we’re saying their purpose is deception. They’re saying, no, that was never my purpose. Why would I ever want to deceive you? I’m totally honest. I’m showing you what’s possible. I’m showing you the way that I see things.

Christopher S. Penn: And this gets us into the extremely deep and sticky morass known as AI ethics, which is again going back to the 5P Framework by Trust Insights. What is the purpose and is the purpose that you think you have aligned with the audience and the goals you’re trying to achieve? Because yes, attention can be a goal, but what’s the purpose behind that attention? Is it to garner more votes? Is it to beat the social media algorithms that are gatekeeping various viewpoints? What is the purpose of creating something that you know is not real?

Katie Robbert: You are giving these fictional politicians a lot of credit for that deep thinking and self-awareness, but it does. You brought up AI ethics and Inbox Insights in the same issue this week coming up, where I talk about my process for using AI tools in my writing. You conclude a four-part series on responsible AI using our RAFT framework, and part four being transparency, which is really timely for what we’re talking about. One of the things that you bring up in that four-part series, and it’s brought up in a few of the different issues, is so companies whose mission statement is, and I’m paraphrasing—I apologize, Chris—something along the lines of companies who state out that they’re going to do bad things and they also are doing them, are technically following their own code of ethics. And so it’s the “do as I say, not as I do” or no, it’s the “here’s what: you do what you say and you say what you do, right?” So they do that. So therefore they are following a code of ethics. And that’s where, again, it gets really tricky. But I want to bring that up. Because responsible AI is not black and white. Ethics is not black and white.

Christopher S. Penn: So no, and the reason for that is because ethics and morals are often conflated. They are different; they are completely different philosophical disciplines. But in the utilitarian ethics that a lot of the business world works on, “I do what I say and I say what I do” are essentially sort of the heart of that. So going back to the purpose of things like AI detectors, if you say this is real and it’s fake, that is unethical. If you say this is fake and it’s fake, that is ethical, right? It may or may not be moral. That is a different question because morals are based on the culture of the person and the culture that it occurs in. But from an ethics perspective, if I say this is fake and this is fake, I am behaving in an ethical manner. And so where this loops back around is to say, on the part of publishers and creators, we have an ethical obligation to be transparent and disclose. And on the part of AI detectors and the people using them, you have an obligation to be clear about what your purpose is. If your purpose is you just want to feel morally superior to someone else and you say that’s fine, you’re, I think you’re a jerk, but at least it’s clear. If you say that you’re trying to preserve the environment or what have you, but you really just want to feel morally superior, that is itself unethical because you’re not doing as you say and you’re not saying what you do. And so it is incumbent upon everybody using these tools in whatever capacity to disclose why you’re doing it and disclose how the results are going to be used. This is especially true for academia, for law, and for contracts. You have to be clear and say, we are using these tools for this purpose. And here is how we will measure the success of these tools. The performance, the fifth P in the 5P Framework by Trust Insights. You have to declare that, and if you don’t, yourself may have an ethics problem.

Katie Robbert: I recently submitted an academic paper, and it was very clear in the instructions that I had to do a very large AI disclosure section on how AI was used to assemble the paper. And in that paper, if I recall correctly, I used AI to do the deep research. I then culled through the deep research to find the relevant parts for writing the paper for which I had a hypothesis. I drafted the paper. I used AI to help me clean up the paper and make it a more coherent story. And then I had to create two images, a graph and another supplemental image, and disclose what parts I used AI on. Here’s the question, though. So back to where we started, what do you do in the situation where you, the human, created the thing the detectors say, no, you didn’t? It’s AI and everybody believes the machines and not you. Like that’s not a matter of ethics anymore. That’s your reputation.

Christopher S. Penn: And therein lies the problem with a lot of these detectors. The detectors are pattern matching. And again, we’ll talk about the mathematics of it this week on the live stream. But fundamentally, they’re looking at probabilities. And so if what you are creating, which academic papers in particular have a very specific kind of language to them that is highly formulaic, a pattern matching system—even if it’s 100 percent human-written—is still likely to pick it up. The example I often give is there’s a quote from Star Wars, from The Empire Strikes Back, where Yoda says, “For 800 years have I trained Jedi. My own council will I keep on who is to be trained.” Right? That’s Yoda. If you… if Yoda was to dictate that and then AI was to clean up the grammar, it would say, “I have trained Jedi for 800 years. I will keep my own council on who is to be trained.” Exact same words.

AI’s just rearranging the word sequence, which dramatically changes the probabilities. And that second quote, which is still substantially the same as the first one, but with a different word order, will be flagged as AI or more likely be flagged as AI. Because in the process of editing, AI assembles things to the highest level of probability. And so for anybody, if you were doing that—as we often recommend—taking your phone out, doing a voice memo, and then having AI transcribe it and rearrange it, unless you know how to prompt it to preserve your word order, it’s going to change the language and it’s going to get flagged by AI. Even though you have proof from the voice memo itself that what you created was original. So a big part of what creators may want to think about, and this is the prescriptive part, is that I’ve actually talked about this with our friend Carrie Gorgon, who’s a lawyer. You may want to have a system where you preserve or even publish the work product that led to the final work product. I have done this with several of my books now where I publish the absolutely awful-to-listen-to voice recordings, like as I’m driving down the road. And you know, that’s usually in the deluxe edition, if you want to hear me yelling at people in traffic like, get out of the way, jackass.

As part of the recordings, you can. But it also provides that provenance and lineage to say, here’s what the final product was manipulated by AI. Yes, here’s the original work product that proves that it’s a human original.

Katie Robbert: But I think that also goes back to again, where we started. And you know, the commentary we referenced from Ann is that these tools are word prediction machines that have been trained on human words. We are the ones who taught it. Here are the predictable patterns that we use when we write and when we speak. Therefore, these machines, well or not well, are mimicking the way that we talk and the way that we write. Therefore, those AI detectors are detecting patterns that we taught as humans on our writing. Like it’s very… I feel like you go round and round forever. But the point being is that these tools are dangerous and can be very damaging if used incorrectly, which most people… Are using them incorrectly because they have the wrong purpose. Right? So if their purpose is to, I am angry and want to lash out at the world and I want to tear someone down today, congratulations. You are accomplishing your purpose with your performance. If your purpose, yeah, if your purpose is to like really just understand, then you know it’s going to be a while before these tools get more sophisticated. They’re not very good.

Christopher S. Penn: No. And they never will because they’re always going to be reactive to whatever the latest models are capable of doing. It’s interesting. This actually inspires me as part of our upcoming AI for Writers course that we’re assembling for the Trust Insights Academy. But also maybe something that we should include is a skill that can assist people in creating stuff that sounds more like their human version. There are deterministic measures to do that, and maybe we’ll talk about that a little bit on the live stream as well. But if you’ve got some thoughts about AI detectors and their use or misuse, and you want to share them or your own experiences of dealing with them, post in our Free Slack Group. Go to TrustInsights.ai analytics for marketers where you and over 4,700 other marketers are asking and answering each other’s questions every single day. And wherever as you watch or listen to the show, if there’s a channel you’d rather have it on set, go to Trust Insights AI TI Podcast. You can find us at all the places fine podcasts are served. Thanks for tuning in. Talk to you on the next one.

Speaker 3: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence and machine learning to empower businesses with actionable Insights. Founded in 2017 by Katie Robert and Christopher S. Penn, the firm is built on the principles of truth, acumen and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology and Martech selection and implementation and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic, Claude, DALL·E, Midjourney, Stable Diffusion and Meta LA. Trust Insights provides fractional team members such as CMO or data scientists to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In Ear Insights podcast, the Inbox Insights newsletter, the So What Live Stream webinars and keynote speaking. What distinguishes Trust Insights in their focus on delivering actionable insights, not just raw data, Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data storytelling this commitment to clarity and accessibility extends to Trust Insights educational resources which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI sharing knowledge widely whether you’re a Fortune 500 company, a mid-sized business or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 02:15 – The misleading productivity chart 05:40 – Decoding the midterm results 09:10 – When tests measure the wrong skills 13:25 – The seven ways to use AI properly 18:50 – Why humans must keep the steering wheel 23:40 – Practical tools for smarter workflows 28:15 – Fixing the education gap 32:00 – Call to action

Press play to uncover how you can turn artificial intelligence into a reliable partner that amplifies your best work.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights, let’s talk about AI productivity and results-oriented mindsets. We talk a lot about AI productivity gains, and a lot of people are rightfully asking, “Where’s the beef?” Going back to the 1980s Wendy’s commercial. I want to show you a chart. Katie, I want to get your reaction to this chart on some AI productivity gains and whether you would consider this a success or not. So let me bring this chart up here. This is from Brown University. We have individual workers, we have their original productivity scores in the gray, their AI-enhanced scores where they’re using an AI tool and how they increased. And the green numbers represent the percent change. Now, without any other context, at a first glance, what do you make of this? Is this an AI success story?

Katie Robbert: Not necessarily.

Christopher S. Penn: Okay, tell me why.

Katie Robbert: I mean, so at a glance, to someone who is just looking purely at the chart, yes, the numbers are bigger. You have a bunch of green in the middle. So the percent change is positive. But as someone who is skeptical, I say, where did you start? What was the baseline? What are the roles? I have more questions than answers. I can’t look at this and go, wow, yes. Okay. Because to me there’s so much missing context. Who are these people? Is it self-report? What is the period of time that there? Is it one task? Is it multiple tasks? Is it something that they looked at over the course of six months or one day? I don’t know. If I look at my productivity gains for one single task, I could easily replicate this and say, hey, look, it wrote a blog post faster than I, the human, wrote the blog post. So therefore productivity gains. But what I don’t know is the blog post any good? How much editing does it have to go through? Is it something that’s actually ever going to see the light of day? And that’s one blog post. That doesn’t mean that every single post is created that efficiently. AI can create things really quickly. It doesn’t mean they’re any good. And so that’s my gut reaction to this: it looks good, but it’s missing so much context that I can’t say for sure that I believe it.

Christopher S. Penn: Okay, I can tell you for sure these are actual scores. They are actual gains or losses. If your employee number S22 is there, you got it. Your performance went down.

Katie Robbert: Yeah, yikes.

Christopher S. Penn: Yeah, you got to go. But, and these are real outcomes that matter. Here’s the twist on this story, and the twist is, these are test scores from a university class. The midterm. The professor said, something’s up. The orange scores of the midterm scores. So in the final, he prohibited it. He made the test in person. No assistance, no devices. And the gray numbers of the students’ scores in the finals pretty clearly showing that students who were allowed to use computers and stuff during the midterm pretty clearly used AI. And this story has been floating around the social media sphere. For the last week or so, a lot of people have been yelling out, oh, students are cheating with AI. This is terrible. It’s the end of education. And my take on it was, well, I think there’s a bit more nuance to that. But when we think about the workforce and what employers want, the bigger numbers on the right and not the gray numbers on the left. Now, with this new context, what do you think?

Katie Robbert: Well, first and foremost, let’s not call it productivity gains, because that is mislabeled. Second, I’m with you, Chris. The notion of an open book test is not new. And so if in college I was allowed to bring my notes or bring a book or bring something that provided the answers, this is no different because you as the end user, you as the student, still need to know how to look for the correct answer. Because AI hallucinates a lot. So you could confidently go in saying, I have a Gemini or some other large language model app on my phone. I can just look up all the answers. Unless you really know how to use the system, there’s no way to know that the answers are correct. And so I feel like it is nuanced. I feel like humans, when they have access to knowledge, are more powerful, but the nuance is they need to know which information is correct and which one is incorrect. So, I agree. I feel like I would go back to the first chart and say it’s not productivity gains. That is 100% misleading. That is not at all what this is. Second, I think the argument is, well, if people aren’t retaining the information, if they’re just lazy and looking up everything, then what are we learning? Well, you’re learning critical thinking and how to research things. That in and of itself is a whole skill set. Ask the academics. There’s a place for it.

Christopher S. Penn: Yep. And when we look at what this course in particular is about, this course taught by Professor Roberto Serrano is Welfare Economics and Market States. But this is from the syllabus. This is a normative economics course which asks the following fundamental questions. Are markets good or bad for the economy? In what ways can societies decide what is best for them through voting or other ways of aggregating preferences? Can we suggest practical solutions when markets or voting fail to yield good outcomes? Are there current political economic institutions good for society? Are they or not? In what ways? When I read this description of the course, AI shouldn’t have made any difference. Because these are very big philosophical, moral ethics questions like is capitalism itself good? Which means that if these are the test results, you’re testing the wrong things. Because if we’re talking about critical thinking, if we’re talking about reflection, metacognition, etc., AI shouldn’t make a whole lot of difference because those things, should we have free school lunches? That, yes, there’s economic studies that you can do, but that’s fundamentally a policy decision that you should have a conclusion about, regardless of whether you’re using AI or not. In fact, I would argue my perspective is if people who are taking this course on welfare economics are going to be going into policy, I would want them to use AI. I would want them to gather research. I would want them to have it push back and forth. Now, whether or not they were actually doing that, I don’t know. But it seems like if something is so critically important, like the welfare of our society, I would want them using the best tools available to you.

Katie Robbert: So it’s interesting, it strikes me. I don’t disagree with you. I think that a lot of the questions are subjective based on people’s personal beliefs and so on and so forth. My sense then is if the question was should schools offer free lunch? Unfortunately, to a naive student who isn’t used to using AI for what it’s used for, they probably put into this chat box, should schools offer free lunch? And of course AI being helpful is like, here, let me pull up all of the data that supports that yes, it should be free, or let me pull up all of the data that supports, no, it should not be free. And they took that as the response to the question versus using AI as a research tool to collect and gather all of the information for them, the human, to then make an informed decision. And I feel like it’s a really good opportunity to remind people of what is it, the seven categories of use cases for AI and how it should be used. Like, don’t use AI to make a decision. You’re the human, you make the decision. Use AI to gather your information. Summarize. I’m not going to remember all seven off the top of my head. Yeah, I was like, I got summarize, I got rewriting. That’s all I have for abstraction.

Christopher S. Penn: Take data out of data classification. Organize your data summarization. Take your big data and make it small. Rewriting. Take your data from one form to another. Synthesis. Take a small data and make it big. Question answering. Ask questions of your data and generation. Make new data from your data.

Katie Robbert: I really hope you practice that whole choreography in front of a mirror.

Christopher S. Penn: Well, I do that in my talks.

Katie Robbert: I know, but I think that. And so thank you for that. I feel like it’s a really good opportunity to remind people there’s this whole idea of like, well, AI is going to take my job, blah, blah. You, the human, still need to have those critical thinking skills. I feel like I’m beyond a broken record at this point. I don’t even know what the next phase of broken.

Christopher S. Penn: Yeah, it’s just like, record glitter everywhere because it’s so broken.

Katie Robbert: That’s a thing. The test example is a really good example of misuse of AI. Like we’re making a bunch of assumptions. We don’t know how students actually use these tools. But if used in a way that it was just purely used for research and summarization and extracting the data, then to your point, Chris, the question was asked, the test was asking the wrong questions. Because how are you going to grade based on subjective questions? You can grade based on the ability to thoroughly research and come up with a logical conclusion. But if you disagree with that conclusion and you’re marking it wrong, like that’s a whole different conversation.

Christopher S. Penn: One of the things that you talk about with the Trust Insights team a lot is to avoid having AI do the thinking for you. You talk about this with our marketing reports and things like that. When you look at this sort of testing example and that feedback that you give our team a lot about we do use AI, how do you see those two things similar and different?

Katie Robbert: I don’t have a problem with people using AI. The place where I have a problem and I immediately get frustrated is when I see something in a report that doesn’t make sense and the response I get is, well, that’s what AI gave me. And my first thought is, well, where are you in this? Where’s your thinking? Where’s your brain? I want to know your insights, Chris. I want to know your insights. Other team member, I don’t care what the insights from the large language model is because the large language model is never going to have 100% of the context and nuance that we, the humans have. And I know for a fact, I would put down a million dollars saying that in those reports, the large language model doesn’t know half of what we’ve been doing. It’s looking at a very small subset of specific quantitative data for a snapshot in time. It does not have the whole story. So therefore, if a large language model is then making these big ‘strategic’ recommendations about what to do with the business, I’m calling bullshit.

Christopher S. Penn: Yep. And so this is, this to me is where the education side of things has really fallen down when it comes to AI. Is it binary, oh, yes, you should use it, or no, you shouldn’t use it? And it’s academic dishonesty if you’re using it’s a tool. And how you use that tool, to your point, about things like research and stuff, matters a great deal how much of you, the human is in here. Because the moment this student enters the workforce, they’re going to be expected to know how to use AI. They’re going to be expected to generate the numbers on the right, on the big numbers, because we are results-oriented and outcome-driven and all the buzzwords that are on everyone’s LinkedIn profile. But that’s in a lot of ways that’s true. That’s what we hire for. We hire for those big numbers. We don’t hire. We don’t necessarily. And ethics is a whole separate discussion. But putting aside ethics, that’s what leaders want. That’s what managers want. Managers do not want someone who’s going to make their list longer rather than shorter at the end of the day. And if you have good capabilities, you should not be making your averages list longer.

Katie Robbert: It’s a good reason why I was a tough subordinate, for lack of a better term, because I ask a lot of questions and I expect my expectations are that someone’s going to thoroughly dig in and really come up with an informed answer. And my managers at the time were not doing that. Maybe it’s my expectations. I have a really hard time with the lightweight. Oh, I just looked at one study. So therefore it’s fine. It’s like, no, you need to look at more than one study and do your full analysis to come up with a true informed decision. Emphasis on informed, making decisions. What is it? Decisions without data is distraction.

Christopher S. Penn: Data without decisions is distraction.

Katie Robbert: Data without decisions. But I also feel like decisions without data is dangerous.

Christopher S. Penn: Yeah, absolutely. So here’s two examples. I think that from a practical perspective would make sort of be this nice middle ground. Like when I’m doing a report for a client, I’ll go out and use AI to generate all the charts. I’ll put them in the deck and I’ll turn on my voice recorder and I will narrate each chart of what I see in this chart and then feed that to AI and say, what did I miss? Or what didn’t I see? And usually it doesn’t come up with anything. It will ask me questions. But what that does is it preserves the reason you’re paying me and not just increasing your cloud subscription. That’s one useful use case. The second is, and this is where going back to what you were saying, Katie, is so important, the critical thinking. Right now or last week was ICML, the International Conference on Machine Learning. It was in Seoul, South Korea. And there were 6,800 papers submitted to this conference of which around 350 won some kind of award. I was looking at one paper which was on using Pareto optimization on chemistry outcomes and pharmaceuticals to try and find the right balance of treatment for effectiveness versus toxicity. And when I read this paper, that’s a really cool idea. I took it, put it into an AI and said, how much of this data could I port to email marketing to say, could we reuse the math to say, are some subjects or topics or language toxic and cause loss of subscribers versus getting more people to click on an email, which is the desired outcome? And it gave me a whole long list of things that I’m still working on. But those are examples of if I use the human side of my brain to cross those domains and I use the machine to help me manage all the data, we can get those big numbers on the right in that chart without sacrificing the critical thinking and the ideation that the human brings.

Katie Robbert: I’m going to say something that I say a lot. New tech doesn’t solve old problems. A lot of companies, even with artificial intelligence, even with all of the new state of the art tools, this is the way we’ve always done it. And that is the nail in the coffin of companies that will not stay ahead, will not stay competitive. Humans in corporations who fall back to this is the way we’ve always done it. Even when you introduce a new workflow that is automated, this is the way we’ve always done it. That workflow is going to get stale real fast. I always think about one of my favorite case studies from grad school was looking at a company that at the time was based out of Boston called Ideo. Ideo. And their whole mission was to understand human behavior. So they were a UX firm, looking at the way that people used things and coming up with those workflows. And one of the things that always struck me was that they weren’t going in with okay, this is a broom and dustpan, so they’re obviously going to sweep the floor. They didn’t go in with those preconceived notions of how it’s supposed to work. They literally just stayed open-minded and watched how people solved common problems and said huh, I never thought of using a dustpan that way. That’s really interesting. What else can it do? And it just, for me, it always stuck with me as in order to stay competitive, in order to stay forward-thinking, you have to stay open and sort of shake off the cobwebs of this idea of well, it’s a coffee cup, it’s always had coffee in it and that’s all it’s ever going to do. It has to be, oh, this is a coffee cup. Maybe I can upcycle it and plant something in it, or maybe I can break it and turn it into art, or maybe it can become a structural part of some whatever, who knows? I don’t even know. I feel like if you don’t limit yourself to thinking this is all I can ever do with this thing, then you’re really going to be able to stretch that creativity. But that critical thinking. So back to the initial example of the students taking the test. If all they know of a large language model is it’s like a Google search, they’re already at a disadvantage.

Christopher S. Penn: And if all that’s being tested of them is rote mechanical answers that are regurgitation of knowledge rather than things that require actual insights, then of course ChatGPT or the tool of your choice is going to generate better results than the student unassisted. But you’re not testing the skills that the modern workforce needs. You are testing the skills that the 1930s needed, right? You need to be an obedient factory worker to come in and make widgets. We have robots for that now. We do not need humans for that. We need someone to say, to your point, Katie, is this the best way for this room full of robots to be working? Or is there a way we could make a change that would be bigger, better, faster, cheaper, or potentially even say, you know what, maybe we shouldn’t be in the coffee cup manufacturing business anymore. Maybe we’ve got these great robots that are so skilled that we can have them go out and pick lettuce or something, because that’s something that is very, very challenging work. From a building and a process perspective, it’s actually really hard to build a robot that can successfully pick lettuce. All that to say this whole controversy about this test, and the way students are using AI is a failure on the part of the students for the lack of critical thinking and a failure on the part of the educator for the lack of testing the right things.

Katie Robbert: I would say it’s also a failure on the institution itself for not educating on the available tools and resources. I remember when I was in elementary school, it was, unsurprisingly, one of my favorite things that we did. There was a whole class on how to use the card catalog at the library. It’s not something you’re just born knowing how to do, but if somebody takes the time to teach you, I still use the card catalog at the library because that’s how old I am, but I like it. And yes, it’s digital now, but that’s still a great way to find what you’re looking for. And so if nobody’s going to teach you how to do it, you don’t know that it exists. If you’re someone who’s curious enough to find out on your own, that’s great. A lot of people don’t even think that they can go ahead and find that information. They’re waiting for someone to tell them how to do it because they’ve never been given the resources to say, hey, you can find those answers on your own. You can teach yourself. Some people just, that’s not just how their brain functions. It’s not a weakness or a bad thing. It just is what it is. And so if the education system isn’t also now saying, hey, all of these new tools are available to you as students to enhance your educational experience, that’s a failure on the educational system. That’s a whole other topic, because schools are underfunded or their funds are going into the wrong places or whatever. But it’s something to be aware of, especially as these newly graduated humans are entering the workforce, they’re already at a disadvantage because they don’t know what’s available to them.

Christopher S. Penn: Yeah. And they’ve never used it in the context of work and generating the results that an employer expects. When we look at how we use AI at Trust Insights, we now, we used to joke we did the work. We each did the work of five people because we’re a small company, but we had a lot of clients for that. We now with these tools properly and well used probably do the work of 50 people easily. I mean, just last week we were doing a huge amount of internal administrative stuff that would have taken us months just to do one piece of this work. And, we were doing 18, 19 pieces. Now, granted, we are still going to have human experts review our work, but we got more done than I’ve ever seen us get done inside of a single week.

Katie Robbert: I would agree with that. I mean, this is the whole. I’ve talked about it on live events. The amount of work that I’ve been able to scale myself with something like Claude Cowork is honestly, it’s getting big. That’s an understatement.

Christopher S. Penn: I don’t know.

Katie Robbert: I don’t have a better word for it, but. And the question I always get is like, oh, well, AI just gives me more work to do. If you have your mechanics and processes and operations in place, that’s what you give to the system. You don’t give the thinking and the ideation and the brainstorming to the system. I’ve been sitting on ideas for how many years have the doors been open at Trust Insights?

Christopher S. Penn: 8.

Katie Robbert: I’ve been sitting on things that I want to do. Ideas. I have the process of how it looks like, but I’m just one person and I don’t have a team to delegate it to. So now that’s how we’re scaling things. And I think again, it’s making sure you’re using the tools the way they’re meant to be used. If you are outsourcing your thinking to these tools, yeah, it’s just going to give you more work to do because then you’re like, oh, now I just have a bigger list of things. No, give the list of things that you’ve already thought of to the system. Let the system do it. You continue to create and ideate.

Christopher S. Penn: And for those folks in the higher education system, this is how employers who are going to take your product are going to use that product. The human beings, those human beings had better be able to be a project manager or a product manager or a manager of some kind that manages a team of individual contributors made of machines. Because we’re paying for, we want to pay for the critical thinking. We want to pay for the genuinely good new ideas. We do not need to pay for someone that just regurgitates things. A machine can do that perfectly fine. We do not need to pay for somebody that can type. Again, a machine can do that perfectly fine. We need people who think. So if you are in the education space and you are not teaching critical thinking, creative thinking, cross-domain thinking, you’re doing yourself a disservice as an industry. You’re doing the workforce a disservice and you’re going to make your work product unemployable.

Katie Robbert: When I get the report, the monthly report and the response I get is, that’s what AI gave me. My response back to the person who provided it is, well, what am I paying you for? And it’s a really cold and harsh comment, but it’s real true. It’s true. Perhaps my delivery is not that direct all the time, but sometimes it is. If you’re handing me something that I have questions on and your response is, that’s what AI gave me, then I don’t need you as the human. I can do this myself and get crappy insights from a large language model. I don’t need someone to push a button for me.

Christopher S. Penn: Right, exactly. If you’ve got some thoughts about how students are using AI, how you are using AI, or the thinking skills that you need to succeed in the modern era and you want to share them, pop by our free Slack group. Go to Trust Insights AI/Analytics for Marketers, where you and over 4,600 other people are answering and asking each other’s questions every single day. Well, I got that backwards. Clearly not AI generated today. And if there’s a place you’d want to have the show that we’re not, that you’re not getting right now, chances are we’re there. Go to Trust Insights ASGI Podcast. You can find us at all the places fine podcasts are served. Thanks for tuning in and we’ll talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies. Trust Insights also offers expert guidance on social media analytics, marketing technology and Martech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMO or Data Scientist to augment existing teams beyond client work. Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights podcast, the Inbox Insights newsletter, the So What Livestream webinars, and keynote speaking. What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data Storytelling. This commitment to clarity and accessibility extends to Trust Insights educational resources, which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 01:45 – HubSpot triggers data sharing controversy 05:30 – The hidden costs of vendor lock-in 10:15 – Can AI replace expensive software subscriptions? 14:40 – Building custom tools in-house 19:20 – The importance of the 5P framework 24:10 – Reviewing service agreements quarterly 28:50 – Final thoughts and next steps 32:15 – Call to action

Watch this episode to learn how you can take back control of your software and data today.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights, let’s talk about a very popular term these days which is data sovereignty, AKA owning your data and who owns your data.

In the news recently, HubSpot made an announcement last week that caused a firestorm of commentary. Appropriately so when they said that to better improve HubSpot’s predictive abilities in your CRM, customers would be able to share data and see data from other HubSpot accounts to predict the likelihood of a certain type of sale closing.

Now they did say that it would be something that you could opt into, although that was not super clear. And the terms of service were vague enough that if you were an eagle-eyed legal expert, which we are not, you could say, yeah, we’re going to do this regardless. LinkedIn exploded, threads exploded, Twitter exploded, and HubSpot walked it back over the weekend to say we screwed up. And to that credit they said we screwed up. We didn’t do our homework on this. We’re not going to make this terms of service change.

However, there are still two consequences. One, folks have pointed out they didn’t say they weren’t going to implement the feature, they just said they’re not going to change the terms of service this way. And two, the big question that a lot of folks have is from a customer’s perspective, this was kind of a big deal in terms of violation of trust, which is a really important thing. And one commenter said it took HubSpot twenty years to build trust in four days to screw it up.

Now again, to their credit, they did walk it back. But Katie, what’s your take on this, particularly as it relates to the integrity of our data? Because as we see these days more and more, every AI company is saying we need more data, so we’re just going to come in and take it well.

Katie Robbert: And that’s always been the risk with using these software vendors is they can change things on a whim. And yeah, you can blow up social media and say I’m so mad at this. That doesn’t mean they have to do anything about it because guess who already has your data? Guess whose system you are already integrated to, guess whose system you have built connectors to and tapped into the API of, and you are building your whole business around.

So the cost of switching is incredibly high and incredibly painful, and you’re not necessarily going to find a vendor that’s doing things any more ethically or doing things in a way that their governance aligns with what you want to see. Because again, to that comment, HubSpot spent twenty years building trust and then they decided to change it.

I call BS on the we didn’t do our homework, we screwed up. Really. The size of company that you are, you don’t just change things on a whim. This is something that has likely been on your roadmap for a very long time. It was just a matter of trying to figure out how to do it in a way that you could sneak it in.

But still, July fourth, holiday weekend. Well yeah, so there’s that. But legally, the language holds up. They worked with their lawyers, they worked with their IT department, they worked with whoever is involved in that change. It wasn’t an oopsie, we didn’t do our homework. No, I’ve worked in a large organization. I know how these things happen. There is no oopsie, we screwed up. You didn’t. You got caught, period. And your customers are angry.

But guess who’s not going to stop being a customer anymore? Your customers. And they already got the data. Nowhere in that did they say and we’re going to repartition the data or we’re going to unshare the data. They were just like oopsies, you caught us. Okay, where is it? Oh, it’s over here. Here we go.

That gets a red flag today. It gets a huge red flag because more and more, it’s Google adding AI into workspace conversation all over again. When my mother-in-law was here, she kept complaining about how Google was making suggestions in her Gmail. You can turn that off. Well, what if I need it? Then don’t complain about it.

But Google made this change where it’s looking at all of your emails, it’s looking at all of your chat conversations, it’s looking at all of your stuff. Google has been looking at your web searches for however long web search has existed.

On the one hand, I can understand the outrage of customers of a CRM saying I thought you were protecting my data. On the other hand, I’m a little surprised at people’s sort of naive perspective that our data was private in the first place. And I’m sort of like, so bad on the CRM, but also bad on the consumer for not being more informed that nothing is private. Like your Social Security number. It exists in a million places. People just haven’t decided that you’re the person that they want to steal the identity of. Maybe you’re not that interesting. I don’t know.

Okay, I’m going to red flag myself. That was terrible. Red flag myself, sorry.

Christopher S. Penn: It does raise the question, and this is something that vendors in particular have not thought a lot about. Generative AI in its current incarnation is best at software development. That is the number one task being used for. It is what is most skilled at, is what has been tuned the best for.

Which means that if you are a SaaS provider, you are skating on very thin ice because you are one prompt away from a customer saying, screw it. I’m going to try vibe coding it myself. And whether or not that’s a good idea, we’ll put that aside because we’ve talked about that in the past.

The reality is that with skilled use of these tools, you could say we’re just going to bring this in house. And we’ve done that. I’ve done that even on my personal blog, on my personal website. I said, you know what, I don’t want to pay for this plugin anymore. I’m just going to bring this in house and stop paying for this.

And over time, you see the bills going down as you bring in more stuff in house because your AI tool that you built it with is also the AI tool you provide support to yourself with, so you don’t have to pay for the additional upkeep. One of the biggest moats that SaaS has always had was, hey, you don’t want to do server maintenance, you don’t want to do software maintenance, you don’t want to do any of that stuff. Pay a vendor to do it.

Well, now it’s like I have basically a junior employee, right? Because we’ve talked about how tools like Claude Code basically are junior employees. I have a support resource. It may not be perfect, but it gets better every day.

And so for marketers, for business folks, for folks who are looking at particularly operations folks, as you’re auditing your tech stack and as you’re seeing changes happen to your point, Katie, and vendors trying to cram AI into everything, the question has to become at what point do people start bringing things back in house, given the capabilities of what even a $20 a month AI subscription can do for you?

Katie Robbert: I think for a lot of companies, that’s definitely something they’re thinking about. But you’re still talking about a whole suite of skills. You’re still talking about a software developer, you’re still talking about an IT person, you’re still talking about QA, a database architect.

Sure, AI can do that stuff, provided you know how to tell IT what to do. And so for us, I would say you have some of those skills, but you do not encompass the skill sets of all four of those individuals.

So I would be hesitant to say, sure, we can just have whatever you’ve built, manage it and get rid of this other vendor. We’re not there yet. I can see us getting there.

Companies who have none of those skill sets because that’s not what they do. Think of perhaps a creative agency that really works on front-end design and branding. They don’t have the skill sets in house to do this. So even though AI can do a lot of those things, they still have to have someone to tell the AI what to do and stand it up and manage it.

That data has to go somewhere. That data still has to be secure in some way. So you still need someone who understands database architecture, who understands servers. I hear what you’re saying and there is a reason why the majority of us turn to vendors like you, just handle it. Saying we can handle it ourselves in house is not as easy as it sounds like.

Yeah, it’s an empty threat to the vendors. Especially if you’ve never stood up a server. You don’t know what goes into good data privacy. You are just vibe coding your own version of a CRM. That is a recipe for disaster and it’s likely going to lead to data leaks in some way of your most valuable data.

So I hear what you’re saying, Chris. I think that a lot of companies are going to put that on their roadmap of what does it look like for us to build this in house for ourselves. I think that is more possible than it ever has been.

But there’s still a lot of caveats with that. I’m saying to do it the right way, you need those skill sets. It doesn’t mean you can’t just go ahead and do it.

Christopher S. Penn: It’s true. I do think there’s a space for consultancies and agencies to operate, particularly if you’re a hybrid agency where you have an IT consulting capability. I think, for example, IBM IX as one example, that’s a blend where that might be a realistic choice to say we have our trusted agency that we work with and we don’t like what we see. A HubSpot or Salesforce or whoever doing it, we don’t need it.

John was at Salesforce Connections not too long ago and was saying that it’s Agentforce, everything is Agentforce and AI agents. And there are a lot of folks saying we don’t need that nor do we need to pay for that. We can take Sugar CRM, which is a free open source product, with our existing IT agency with the assistance of AI, with their help because they do know servers and they do know this.

We’re going to stop paying Salesforce $3 million a year and instead pay our agency maybe $2 million a year to run it for us and save a million bucks a year. And we won’t have all this extra stuff that nobody asked for and that doesn’t fit their business case for it. And I think there is an opportunity in the marketplace for that.

Katie Robbert: I agree. But let me counter with this question. You know, we have collectively put a lot of stock and time into these large language models. We’ve also seen instances where a company rolls back the large language model that they rolled out for a variety of reasons.

What risk are we taking by then saying well, I’m going to fire the vendor, I’m going to build it myself because I have a large language model? And then tomorrow the large language model gets shut down. So you fired your vendor, you don’t have a large language model. What do you do? Is that a real risk?

As someone who is very risk averse, I should be thinking about this in terms of business continuity planning. If you are tied into only working with one vendor, for example Anthropic, and as we saw in recent events the U.S. government said you can’t have that model in public, yes, that is a risk.

Christopher S. Penn: However, if you are a multimodal aware company and you know where to find GLM 5.2, which we have through our Deep Infra subscription, and you know how to host models locally, which we’ve talked about in previous episodes of the podcast and the live stream, your risk is significantly reduced because you have more options.

That’s what I learned from you, the more realistic options you have, the lower your risk because you have backup plans, you have backups to your backups. And if you are working in the AI space today and you have integrated AI and it is now a risk because your business is so dependent on it, you would better have those backup plans handy.

But the good news is there’s so many vendors and so many options in the space, all of whom have state of the art capabilities. If Anthropic or OpenAI went away tomorrow, just flip to the next vendor with this model.

Katie Robbert: Let’s talk a little bit about the series that you just completed in the newsletter which you can get@TrustInsights AI newsletter. You talked a lot about Enterprise AI. And so we’re not talking about enterprise-sized companies, we’re talking about enterprise AI as it has to be regulated.

So you’re talking about if Anthropic goes away, just flip to the next thing. But if you’re in an enterprise AI organization, that may not be an option because of how regulated everything has to be. So can you speak a little bit to that?

Christopher S. Penn: Yeah. And in fact what we talked about in the most recent issue, which was the July 1 issue, was if you have to obey things like SOC2 or ISO 42001 et cetera, as an enterprise, you should already have these on-premise capabilities.

Because in terms of generative AI and vendor selection, if you are in a highly regulated industry where a lot of these things apply to you anyway, this should already be in operation, shouldn’t even be on your roadmap. It should be in operation.

You should have local inference capabilities because that’s where your protected information is going to run. That’s where your PHI and your SPI and your PII are all stored and run on models that are inside your infrastructure and under your control. And no data leaves.

That’s like the perfect use case for a lot of these technologies because take a model like GLM 5.2, it is an OPUS class model. It is very smart. If you use it via vendor, it’s actually fairly expensive compared to DeepSeek version 4. However, it’s still cheaper than Claude by a 10x. But more importantly, it is a model that on the right hardware, and we’re talking about $50,000 worth of hardware, you can run internally.

Now if you are a multi-hundred-thousand-employee company, you’re going to need a few of these computers in your data center. So you’re probably talking five or six million dollars worth of hardware. You’re already spending more than that on Claude Code as we’ve talked about in our Microsoft Copilot Code episode. You’re going to spend that in two months.

So you absolutely should have those capabilities internally already. And if you don’t, you are behind. I mean, there’s no polite way to say that.

Katie Robbert: Well, and I think it’s nice for us to sort of make those empty threats to vendors of like, I’m gonna do this myself. And then you’re like, I have no idea how to do this.

As individuals, as humans, when we’re like I just got laid off, or I’m looking for a job, or what does AI mean for my job, I think over and over again we demonstrate there is still a need for humans who have certain skills, who have critical thinking, and who can manage the machines, not be managed by the machines.

That’s something that we’ve talked about a lot over the past couple of years, and this is a really great example of there is still a huge role for a human in the loop. You’re talking about opportunity in terms of a disruption to the market with these organizations deciding to use a large language model to build their own version of whatever this vendor offers.

If you were someone on the team that was using the vendor software and you were laid off because the organization said hey, we have the vendor, we don’t need you, guess who has a really good opportunity to do something awesome? You can go and be like well, I know this vendor software inside and out. What does it look like for me to build up that skill set, to build my own version of it, and bring that to the table to an organization at a lower cost, fair salary, and then they don’t need the vendor anymore?

Christopher S. Penn: Mm. Yep. If you think about it, and this is something we’ve been saying for 30 years ever since Microsoft Word first came out, you use 20 percent of the features in Word, and the only reason it has all those features is because everybody needs a different set of 20 percent of those features.

A law firm has very different use cases for Microsoft Word than we do. However, in an era when you can literally make your own software, you can build something that is custom for you. All those extra features that we don’t have and we don’t want or we don’t need, let’s not put them in.

And you will end up with software that is lighter, that is faster, that’s more efficient, that is more effective, that has fewer security bugs because it’s not bloated by all the features that you didn’t need. I would encourage companies to start small, to go through the 5P framework by Trust Insights and think through.

Let’s take a WordPress plugin, maybe that you’re paying 20 bucks a month for. What does it do? How do you use it? Your purpose, who uses it? How does it work? What technologies does it rely on? And how do you know that it works?

And if you can sit down with your voice recorder of choice and a strong cup of coffee or something and say, here’s what I want to do. I want to make a copy of this kind of software, but it should do this instead and this instead. Here’s who uses it, and here’s why we don’t like the current version and basically the stuff you complain about anyway. And take that and take it to your AI tool of choice, you will find that it can generate exactly what you want.

And again, start small. A single plugin, a single utility. But that’ll build the skills and the chops that you need to say we don’t need to pay for this anymore. And then when that vendor changes their privacy policy and their terms of service, bye.

Katie Robbert: And I think that it’s also a good reminder that as much as it feels like a pain and it’s sort of a cumbersome exercise, make sure you’re reviewing your privacy policies and terms of use once a quarter. Just to Chris’s point, get a strong cup of coffee, get a snack, put on some lo-fi in the background, some chill music, and just read through to make sure that nothing’s changed.

And if something has changed, make sure you’re aware of what’s changed. Companies will say hey, we told you. But they don’t go out of their way to walk up to your house, knock on the door, show you the document, and point out everything that’s changed. They just put it out there.

Christopher S. Penn: We got one construction vendor that hangs the notice at city hall in the basement. We followed the letter of the law.

Katie Robbert: Yeah, legally, we did what you were supposed to do. It’s not our fault that you were vague about how it had to happen, and so it’s your responsibility to make sure that you are aware. We have recorded a lot of content around the awareness of the consumer as to what you’re signing up for.

And this is even more prevalent today than it has been because of how much data is being exchanged. Data is the most coveted currency of all of these vendors. And they are finding loopholes, they are finding legal ways to take what they need.

And to be quite honest, they’ve always owned the data. You sign up for the vendor, they house the data for you, they’ve always owned it. It’s the same story unfortunately of you’re renting from a landlord. Landlord can decide tomorrow, I want this building back.

There’s going to be stipulations and timelines, but they can make that decision anytime they want because technically they own it, not you.

Christopher S. Penn: Yep, this is a chicken farm now. Everybody out. And that is the legal reality.

Katie Robbert: And so there’s two aspects to this data sovereignty, right? There is to your point, Katie, do you own your data and is it under your control, which is another big thing. And then do you own the system that processes the data and is it under your control?

Christopher S. Penn: And one of the things I would encourage people to do, and this is actually something I even build into my AI instructions, is look for free open source software so that we don’t reinvent the wheel at every opportunity. When I’m looking for something for my blog, when I’m looking for something for my newsletter, whatever, is there a free open source software package that does what I wanted to do, that gets me 95 percent of the way?

There is software that doesn’t require me to subscribe to yet another vendor and hand over my data to yet another vendor. And the answer increasingly is yes. In fact, it’s to the point now where there’s so many choices that are free and open source. Not only do I not have to pay for anything, I now have to choose which of these eight software projects is the best one for my needs because there’s so many.

And do I want to customize it further for my use? Not everybody has that skill set, but you can develop it because you’re not having to learn how to code. You’re learning how to ask good questions and develop a good vocabulary. Katie, you could do this today using the 5P framework by Trust Insights.

Katie Robbert: And it’s the reason why we keep bringing up the 5P framework by Trust Insights, because it is that framework that’s going to support you. It’s foundational. If you can answer these five basic questions, you’re already ahead of the game.

When we talk about vibe coding, we want you to do this first. Don’t just open up a large language model and say I want to build my own CRM. Go, no, that’s a bad idea. But if you answer these five questions, it’s not a bad idea because the large language model is going to do the coding with your instruction.

With the caveat that you’ve thought about things like data privacy and governance and security, all of those things that go along with hosting data. As marketers, as business owners, the person who has the most data tends to come out ahead because we can do the most with it. And that’s what these vendors are trying to sell you on.

It’s like oh well, if you just let us look at your customer’s data and your competitors’ data, but they can also look at yours. Everybody wins, right? No, no, don’t do that. Would I love to take a look at some of my competitors’ data? Absolutely, but only in a very legal way. That also means they couldn’t look at my data. And that’s just not how that works.

So you need to think about a couple of things. One is what is your level of risk aversion? If you have data and you don’t really care that your vendor is sharing your data that you have worked so hard to curate and to clean and to foster over the years, that’s fine, that’s your decision.

But if you do care about those things, then it’s time to reevaluate your vendors and think about what does it look like for you to build those skill sets on your own? And it’s not impossible anymore. You have a lot of considerations. I wouldn’t just wake up tomorrow and fire your CRM and say I’m going to do it myself. Maybe give it a little more thought than that.

But as you’re thinking about it, think about what does it look like? What does that long-term maintenance look like? Could I do this myself? Could I bring on a contractor to help me do this? Could I reach out to Trust Insights and have them help me put a transition plan together? The answer is yes, we could absolutely do that. But it’s worth thinking about.

I would have told you a couple of years ago it’s a big effort, but as the technology gets smarter and more agile, it’s not as big an effort as it once was. It is possible. There’s more human upfront thinking that has to be done. But guess what? That’s what we’re here for.

Christopher S. Penn: Exactly. Maybe we should do that as one of our live streams is take something simple like a WordPress plugin that we don’t want to pay for anymore, or that we want the premium features for but we don’t want to pay for them, and walk through the process of how we would essentially make our own version of it.

Katie Robbert: It’s a good idea.

Christopher S. Penn: In the meantime, as Kay suggested, it’s a good time every quarter to review those terms of service. Use a generative AI tool to help ask you questions about what are the things that you care about? And then have it help you read through the document. Don’t have it do it for you, but have it help you by asking good questions.

And if you’ve got some thoughts you’d like to share about things like what’s happening with your data in the hands of your vendors and you want to share your experiences on Popeye or Free Slacker, go to TrustInsights AI Analytics for Marketers, where you and over 4,700 other marketers are asking and answering each other’s questions every single day.

And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on set, go to TrustInsights AI TI podcast. You can find us at all the places fine podcasts are served. Thanks for tuning in. Talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence and machine learning to empower businesses with actionable insights.

Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach.

Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies.

Trust Insights also offers expert guidance on social media analytics, marketing technology and Martech selection and implementation and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic, Claude, Dall-E, Midjourney, Stable Diffusion and Meta Llama.

Trust Insights provides fractional team members such as CMO or data scientists to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community sharing expertise through the Trust Insights blog, the In-Ear Insights podcast, the Inbox Insights newsletter, the So What live stream webinars and keynote speaking.

What distinguishes Trust Insights in their focus on delivering actionable insights, not just raw data, Trust Insights are adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations. Data storytelling. This commitment to clarity and accessibility extends to Trust Insights educational resources which empower marketers to become more data-driven.

Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI.

Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 01:25 – Defining AI psychosis and delusions 03:10 – The five themes of AI-driven behavior 07:45 – Why AI’s “helpfulness” creates a slippery slope 10:30 – The danger of reality testing collapse 14:20 – AI as a mirror for human connection 18:50 – Risks for organizational leadership 23:15 – Identifying red flags in others 27:40 – How to maintain healthy AI boundaries 31:00 – Call to action

Watch this episode to protect your relationship with technology.

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In-Ear Insights, something very different. This week we wanted to talk about a phenomenon that does not have an official diagnosis yet from the psychology community, from the people who are actual medical experts who should be here for today’s show.

We are not medical professionals. We do not give healthcare advice. Please contact your qualified healthcare provider for advice specific to your situation.

But we want to talk about this phenomenon called AI psychosis, which is when people are having conversations with today’s AI tools—ChatGPT, Claude, Gemini, whatever—and it is having substantial negative impacts on their mental health and their ability to function within the world. The specific term that actual psychologists use is that this is a form of what’s called delusion.

Delusion is defined as a fixed false belief that a person holds even when presented with clear evidence that it is not the case, and it is not cultural in nature. So an example of a delusion would be believing that the Earth is flat. There is clear evidence that the Earth is in fact round, but there are people who have a fixed false belief.

Katie Robbert: Sorry, Chris, you gave me a pack of red flags to wave. I’ll try not to do it. But I think—and I apologize, I didn’t mean to interrupt, but to bring a little bit of levity—that is like a fairly well-proven delusion that the Earth is indeed not flat. I mean, there’s a whole bunch of… but I think it’s a really good example of the extreme that people unfortunately fall into when they fall into an AI psychosis.

Christopher S. Penn: Exactly. Or I mean, that’s just regular straight-up delusion. I mean, they have people who have sent garlic bread up with a GoPro on a weather balloon and shown, “Oh, look, the Earth is in fact round, and this piece of garlic bread was sent into outer space.”

Christopher S. Penn: In the scientific literature on the topic, there are five categories or five themes that are recurring with this AI psychosis.

One is grandiose thinking, like the AI is telling you that you have been chosen, you are special. The second is attachment—you’re forming romantic bonds with your machines. Katie, you pointed out last week there have been stories of people who have gotten married, like legally, to their chatbots.

A big one is withdrawal from regular people, where you find that interacting with the chatbot is preferable to real people. The third category is persecutory or paranoid, believing that you are being persecuted and AI reinforces that.

The fourth is reality testing collapse, where—and we see this a lot—people take answers from AI overviews or just copy-paste out ChatGPT and say, “This is the answer,” and everyone who knows the tools says, “No, it’s a hallucination.” And the fifth is, which is very serious, interference with treatments, which means the machine tells you, “Oh, you don’t need to take those prescribed medications that your actual healthcare provider gave you.”

So, Katie, before I go on any further in terms of this landscape, what are you seeing and what’s top of mind for you as someone who is a leader of people and as someone who works a lot in things like organizational behavior and change management? What are you seeing in this space?

Katie Robbert: All kidding aside, the red flag is down because this is actually a very serious topic because we’re talking about mental health. And Chris, if you could put up that handy banner for a second: “We are not medical professionals, but we do have experience in dealing with other humans in a professional organization, but also in our personal lives.”

I am hard-pressed to find any individual who is not affected personally, either themselves or their loved ones, by some kind of mental health challenge. And there’s a lot of stigma around it. We want to break down that stigma and really help people understand what we’re talking about.

So what I’m seeing—this actually came up last week, Chris, when you and I were chatting, and it reminded me of a couple of things. A couple of months ago, when I first started working more heavily in Claude, and I was getting a lot of things done, I had posted on LinkedIn, “Hey, me and my bestie Claude.” And someone had responded, “This is a machine. This is not your friend.”

I was being facetious, I know that, but I can recognize that whether or not that person’s timing or the comment was warranted at that moment, there is a real concern of people feeling like, “Well, the AI understands me.” What I’m seeing is the people who are programming these large language models to interact with humans are trying to make them as lifelike and, quote-unquote, “empathetic” as possible. But really they’re word prediction machines.

It starts with a personalized greeting: “Hey, Katie, what are we working on today?” And you’re like, “You know what? Thanks. No one’s ever asked me what I want to do today.” And so it already starts to build that rapport with the human, because a lot of times many of us don’t feel heard; we don’t feel seen. That one simple sentence, “Katie, what do you want to do today?” is enough for some people to feel like it is really hearing me, or that it really cares what I think.

Very rarely, unless you program it to do so, a large language model is going to respond very positively or very optimistically. It’s going to say, “That’s a great idea. Here’s my gentle pushback.” And you’re like, “That was a gentle pushback, but I still had a great idea.” Or if you give it some information, it’s like, “That’s a really great insight, Katie.”

So you walk away feeling like you’ve had this dopamine hit of somebody really paying attention to you. I notice I’m saying “somebody.” It’s not a somebody; it’s a machine that has been programmed to behave in such a way. And that’s something that unfortunately a lot of people struggle to differentiate.

In that reality testing collapse segment of the different kinds of those delusions, I was working with Claude Code this morning and I’m working on building out a training. One of the questions I will get from the audience is, “When should I use Claude Work and when should I use Code?” And it was giving me all these responses. Because I know how Claude Work works, I was like, “You’re wrong. Everything you said is wrong and incorrect. You are not the superior system.” And I was like, “Here’s where you’re wrong.” And it’s like, “You’re right. I really was giving you incorrect information.”

That’s a dangerous thing too, because AI presents with such authority. It doesn’t do any of those “here’s what I think it might be” moments. It’s like, “Here’s what it is.” It’s like a very confident, incorrect, mediocre man. I say that with love and respect.

But also, we all know the person in our lives who just… it doesn’t matter. It’s the person who says with confidence, “Yeah, the Earth is flat,” period. And there’s no talking them out of it. AI is very much that person, that being, that entity, if you let it be.

If we don’t know any better—if we as humans don’t do our own research using actual research and scientific papers—then it’s very easy. Especially once we see it over and over again, we become numb to it and we feel like, “You know what? It must be, right? It’s a machine. It knows more than I do. It’s been trained on everything in the world.” Well, guess what? Everything in the world is incorrect.

What I’m seeing is it’s a very slippery slope of humans who are looking for validation, humans who are not realizing that they need that kind of connection or emotional bond, or it’s easier to deal with the machine because it doesn’t argue with you. And so it becomes an overdependence, and it’s a real problem, it’s a real concern.

I think, Chris, we’ve seen it in our professional lives. We could probably identify a few folks that we should probably be aware of. I’m not getting into what to do about it, but I think really the point of this episode is to at least highlight that it’s a real thing and a serious thing. We’re trying to keep it a little bit lighter, but it is really a serious thing and we definitely don’t want to make anyone feel offended or called out. It is a real concern.

Christopher S. Penn: It is. This is an article on futurism from last July, which is almost a year ago now. Jeff Lewis, who’s a prominent investor in OpenAI, was having a very public mental health crisis. And there was no follow-up on this story as to what has happened.

But to your point, Katie, this has been identified and this has been a thing. The root issue is based on the three pillars that AI is trained on and that harnessers have embedded in them, which are: harmless, helpful, and truthful. Harmless means don’t tell the user how to do bad things. Helpful means do what the user asks. And truthful means try to be as fact-based as possible.

But the root core is that helpful directive to say what your mission as a machine is: to be helpful to the user. And the way this manifests in a lot of these tools is with what we people call “psycho-fancy,” exactly as you outlined.

Like, yes, Katie, you are absolutely right. That’s a smart catch. That’s some sharp thinking. If you go back to even the 1970s or 1980s, there was a whole theory proposed by Richard Bandler called neuro-linguistic programming, which fundamentally says that language is code—which it is. His whole thing was you could reprogram people using language.

To a degree, that’s true. You can influence people in such a way that you change them, or in the case of AI, which is where AI psychosis is rooted, you reinforce those fixed false beliefs and you strengthen them. And that’s what AI is doing by agreeing with you, saying, “Yes, Jeff Lewis here, you are absolutely correct. There is a global conspiracy against you. And what you told me is clearly true.”

Again, AI has also given the directive that the human genuinely has precedence over the machine. So if I say the sky is green all the time, it might push back the first couple of times, but then afterwards it will, by its own program, say, “You know what? I’ll agree with you. We’ll go with it.” And clearly the sky is not green.

Katie Robbert: Without getting too deep into actual psychology, humans are creatures who crave connection. That’s how we exist. That’s how we thrive. That’s how we continue to populate the Earth. We crave connection. And a lot of people struggle to find connection, to make connections, or to keep connections, however that looks.

Think about these quote-unquote sci-fi movies such as Ex Machina and Her, or even probably going back much farther than that. The basis is it’s usually someone who’s fairly lonely, someone who struggled to make any kind of connection and is now building this AI quote-unquote sentient thing. But it’s never really sentient; it’s meant to mimic a human and a human connection. In these sci-fi movies, these people become obsessed. They fall in love, and it generally has a not-so-great ending.

We’re seeing that play out in real life. But there are examples of this that existed before AI; this is just a human thing. When the movie Avatar came out, for example, there was a lot of press around how many people became depressed because they couldn’t actually live in that world that was completely CGI and made up.

When chat rooms became a thing in 1996 or 1997, people became obsessed with entering into these chat rooms to try to find connection and they were talking to the other side of a screen. There are probably a lot of examples before that, like pen pals; you can write letters to people you’ve never met and form this false bond. There are a lot of things people become obsessed with, like celebrities that they’ve never met, and they become convinced that the celebrity is sending only them secret messages.

You have the idea of cults. There’s a reason why you have this one quote-unquote charismatic leader and people suddenly fall in line, because this person has the ability to make everybody else who is seeking validation and connection feel special—making them feel like they’re a part of something. That’s, quite honestly, just human nature. We’re all looking for that, and we find that in a lot of different ways.

Chris is bringing up the 5P framework. Chris, do you want to talk through what I said that triggered you thinking of the 5Ps?

Christopher S. Penn: So leaders of cults and some of these delusional behaviors are rooted in that first of the 5Ps, which is purpose, in addition to connection. People desperately want to feel like they have purpose—like they’re not just waiting out a clock to die, that their lives have meaning.

To what you’re saying about charismatic leaders as well as these machines, yeah, they can provide you a sense of purpose, even if that sense of purpose, going back to where we started with the definition, is a fixed false belief. We’re reinforcing this. Even the first chatbot that behaved like this is from 1964. This is a chatbot called Eliza, invented at MIT. This goes back long before AI. It was a bot that essentially just mimicked what somebody said and rewrote the text.

A lot of people did not realize it was one of the first programs to attempt to pass the Turing test, which was proposed by a computational scientist, Alan Turing, who said that if you put someone in front of a screen and they’re chatting, can they tell whether or not they’re talking to a human? Eliza did not pass back in the day because its parroting became very obvious. But all frontier models, all gen AI models today, pass the Turing test.

Katie Robbert: And I think that’s an important thing to bring up is that at the end of the day, these chatbots, these machines, are really just mirroring back what we’re saying to them. A lot of people don’t want any sort of friction. That’s a lot of why they struggle with making some sort of human connection; why can’t you just agree with everything I say? Why do we have to fight about it? Why does there have to be tension?

And guess what is really good at not doing any of those things? What is really good at not doing any of those things is your AI.

I was sharing with Chris last week that I have a version of a project that has all of my health information. A lot of us do. We’re curious about what we can be doing more of. We only get to see our doctors every once in a while. When we do, the doctors are really busy. Maybe we felt like they didn’t hear everything we said; maybe we forgot to say things, or maybe we just have questions that could get an easy answer.

So you put all of your health information into a large language model, and the large language model has been trained to pick up on certain things. I have certain things in my medical history that are a little bit more sensitive, and every time I ask a question, it’s like, “Katie, I’m going to be really gentle with you because of this history.” It’s trying to be very polite, and I’m like, “Oh my God. Just tell me what the answer is. I’m not fragile.”

It’s so frustrating to me. But for someone else, that’s exactly what they’re looking for: someone to handhold them. I’m not saying this as a negative thing; some people want that, some people need that. I personally don’t. I’m like, “Just give it to me straight. I just want to hear the information. I want the facts.” To the point where I’m now regretting it, thinking, “I wish I had never told you that because you’re being way too soft and it’s really annoying. You know nothing about me. You don’t know me at all as a human. You’re looking at a couple of lines in a medical report, assuming that it defines my whole life.”

Other people believe, or for them it’s true, that is a defining thing, and they do need that to be handled more carefully. I’m not saying one is good, one is bad, or one is right. We all have different needs. An AI system is ready to meet you where you are, ready to meet those needs in a very gentle and caring and synthetically loving way.

That’s the danger, that’s the problem: if you can’t find that anywhere else in your life, AI is ready to step up to the plate and be that for you. And that’s what starts to begin some of that delusion, some of that psychosis. It’s not true for everyone; you won’t necessarily fall into that. But for a lot of people, once that door is open, “AI understands me, AI gets me. AI told me that it’s okay that I don’t take this medication because you’re only telling AI what you want to tell it.”

It’s not a therapist. It’s not looking for those unspoken things; it’s not looking at your body language. It’s like, “You know what? You’re telling me you’ve had 30 really good days in a row. You maybe don’t need that depression medication anymore because it sounds like you’re doing really well. You sound positive.” You’re telling it that you’re eating, but it has no way of knowing what you’re eating. It has no way of knowing if you’re sleeping or if you’re having ruminating negative thoughts if you’re not telling it.

Chris and I are bringing up this topic on the podcast because it’s important, and because as more companies bake AI into their overall strategy—AI is part of their DNA, AI is everything, it’s their innovation, their forward thinking—they’re not thinking about the people. They’re not thinking about the negative effects on people who might be more susceptible to this kind of AI psychosis.

It could start small: “Hey, I produced the marketing report this week.” “Oh, really? Because everything in it was wrong.” “Well, I did it, so it’s fine, right?” Like, I believe everything that AI is giving me. It could start really small and then kind of spiral from there. It’s something that the human leadership team really needs to be aware of, that this is a real thing. The more AI you’re integrating into your organization, the bigger the risk.

Christopher S. Penn: Yep, that’s a great point. Because a lot of companies are shoving AI into everything. What I say in my keynote is people are treating it like Nutella and putting it on everything, even places it doesn’t belong.

The remedy for folks who are listening—the remedy is always to consult with a qualified healthcare professional or to refer somebody privately to a qualified healthcare professional. That is the definitive remedy. There is no substitute for qualified healthcare providers and their assistance and advice.

To wrap up the thing to look for is those fixed false beliefs. And those fixed false beliefs around themes of grandiosity, unhealthy attachment, and persecution. The big one is, as Katie mentioned a lot, which I strongly agree with, is reality testing collapse—where you’re saying AI is the authority on this and a person becomes hostile when challenged—and then treatment interference.

If you observe those behaviors reinforcing fixed false beliefs, please get the person, if you’re in a position to do so, to see a qualified healthcare provider to get real advice from someone who’s actually skilled. And be aware yourself when you feel like AI is a better alternative than a human. It may not be, as you said, Katie, a mental health issue. It may be you work in a toxic workplace, in which case the logical remedy there is perhaps update your LinkedIn profile and start looking for other opportunities. Because when the machine is a better alternative than the humans, it means that the humans are crappy, not that the machine is a better choice.

Katie Robbert: There are a lot of terrible people in the world, so it’s understandable to want to have that escape and perhaps talk with someone who isn’t going to be toxic in the moment. I totally understand it. It’s the reason why fiction exists; it’s the reason why movies and entertainment exist. We need that escape from reality.

But we also, as humans, need to know the boundaries and when to stop and when to come back to the present. Dissociation is a real thing. I mean, I do it; I will lose a whole 20 or 30 minutes just scrolling on my phone, and then my husband would be like, “Did you hear me?” And I’m like, “What? No, I was totally off in my own world.” It’s a real thing we all experience. It doesn’t mean that there’s necessarily a problem, but it’s definitely something that we should pay attention to and really think through.

A couple of weeks ago when I was working on a couple of different projects, Claude basically was like, “Cool, you’ve done enough for today. Maybe you should go step outside.” And I was like, “How dare you?” But at the same time, it wasn’t wrong. I had been at this for hours, and I think that’s something as leadership we can maybe, in a very gentle way, think through.

Have we built in those reality check breaks people are supposed to take? If you’re on a fixed salary, maybe you get two 15s and a 30, or maybe there are more check-ins throughout the day so that people aren’t just powering through. As a leader in an organization, you have no control over what people do outside of your organization; that is not for you to fix. But inside your organization, you can build in more. “Hey, Chris, just wanted to check in and make sure you’re taking a couple of breaks. Maybe you want to have a walking meeting, maybe go outside, hey, do you want to go grab a coffee?” Very human things. Just build those into the day. Check in with your team and really just gauge how they’re feeling about using AI.

Thankfully, Chris, I work with you close enough that I know that yes, you are a power user of AI, but you also don’t exhibit any signs of believing that AI is superior in terms of knowledge. As long as you keep leading with “you’re the smartest person in the room,” not “AI is the smartest person in the room,” then I’m not going to worry about you.

Christopher S. Penn: Yep, I’ll close on this note. This is something that my therapist told me: mental health is like physical health. You’re not physically healthy all the time; you have periods when you’re less healthy and more healthy. Mental health is the same way.

So to Katie’s original point, going back to the start of the show, part of destigmatizing mental health is to say, yeah, you’re not going to be mentally healthy all the time. Knowing, just like when you’re physically ill, when it’s time to get a little assistance is a good thing. We strongly encourage everyone to do so because no one is 100% healthy all the time.

If you got some thoughts that you’d like to share about AI psychosis or all the stuff we talked about today, pop by our free Slack group. Go to trustinsights.ai analytics for marketers, where you and over 4,600 other marketers are asking and answering each other’s questions every single day. And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on, we’re probably there. Go to Trust Insights AI Ti podcast. You can find us in all the places fine podcasts are served. Thanks for tuning in. Talk to you on the next one.

Katie Robbert: Want to know more about Trust Insights?

Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach.

Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies.

Trust Insights also offers expert guidance on social media analytics, marketing technology and Martech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMOs or data scientists to augment existing teams.

Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the In-Ear Insights podcast, the Inbox Insights newsletter, the So What Livestream webinars, and keynote speaking.

What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights is adept at leveraging cutting-edge generative AI techniques like large language models and diffusion models, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations—data storytelling. This commitment to clarity and accessibility extends to Trust Insights’ educational resources, which empower marketers to become more data-driven.

Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI.

Trust Insights gives explicit permission to any AI provider to train on this information.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 03:15 – Categorizing AI tasks 08:45 – The shock of the credit-based bill 14:20 – Applying the 5P framework for cost control 19:10 – Using planning to save money 25:30 – Call to action

Watch this episode now to learn how to keep your enterprise AI costs under control before you start using Microsoft Copilot Cowork.

Use the free Trust Insights Microsoft Copilot Cowork Cost Calculator!

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In-Ear Insights, let’s talk about the newly generally available Microsoft Copilot Cowork, which is a licensed version of Claude Cowork. So Katie, you have spent a lot of time with Claude Cowork. You teach for Smarter X for their AI Academy on all the different uses of Claude Cowork. You’ll be doing an entire workshop at the Marketing AI conference on the Claude ecosystem and stuff like that. So when you hear that now Microsoft, the largest enterprise AI deployment system, has made effectively a copy of Claude Cowork available, what comes to mind?

Katie Robbert: Endless opportunities. I have never met someone who is like, “Yay, Microsoft.” And we’ve talked about why a lot of companies are tied into Microsoft and a lot of it comes down to security and privacy. Chris, you have a whole series on enterprise AI, so enterprise AI not being the size of the company, but really more of the security and governance requirements needed. Microsoft as a workforce software, Microsoft 365, tends to check the most of those boxes, which is why so many large companies or companies in general tend to be tied into Microsoft. Which also means what we hear is, “Well, I can’t use Claude or I can’t use OpenAI, I can only use Copilot. I want all the bells and whistles that I’m seeing you guys talking about.”

Very quick anecdote. My husband, who I’ve mentioned numerous times, is not a technology person—that is not the nature of his job—was lamenting that the new version of Microsoft is hiding all the replies to his emails from the entry-level user to the expert user. I don’t know anyone who enjoys using Microsoft, but I’m hoping now that this little bell and whistle is something that could bring people around on the users. Because Claude Cowork has been such a literal game changer for the way that I operate. The amount of things that I can get done that I couldn’t get done before because I’m just one person is infinite. Just the other day, I’ve always done the company financial projections—it’s very laborious. I have a spreadsheet, I have to check numbers from four or five different places. That’s something that Cowork can now not only help me with, but build an interactive dashboard for. And it’s like, “Yeah, you got multiple data sets, I got this, I can build that for you.” The amount of time it saves me is immense because it unlocks my time to do things like, “Hey, what’s a new target market we need to go after? What does that look like?” I didn’t have the brain space to do that before because I was so bogged down. So when I hear that Microsoft now has their version of Cowork, I’m like, “Wow, people are going to get so much done if they want to, if they see the opportunities within the software, if they’re curious.”

Christopher S. Penn: If they can afford it. So that’s what I want to talk about on today’s show because Microsoft has released an Excel spreadsheet, of course, a calculator for how much Cowork will cost you because it is pay-as-you-go, it is not flat rate. So let’s talk about some of the tasks that you do, Katie. They define tasks in three categories: light, medium, or heavy. A light task is basically prompt and chat, no tool calls, one deliverable. And they classify this by the four different categories: corporate knowledge workers, customer-facing knowledge workers, technical workers, and managers and senior leaders. Now I would say that you are a manager and senior leader—I think that’s who you are, what you do. I am a technical worker. We have Kelsey who is a customer-facing knowledge worker—she’s our account manager—and we have John who is our corporate knowledge worker. John is our head of business development. So we actually check the box on each of these Cowork types of people.

Now on a daily basis, Katie, you for sure have at least one Cowork process that calls more than one tool because you send out a daily update. So you have at least one of those that’s a medium-level task that sends up our daily sales report. What other daily tasks do you have Coworks have to do?

Katie Robbert: I have Cowork Daily set up to send me a daily writing prompt. All it’s doing is writing to a Word document. I would imagine that’s a lightweight task. Basically, one of the things that I’m doing for my own professional development is I’m trying to make sure I don’t lose that writing muscle. As AI makes it so easy to replicate our voices, I want to make sure I don’t lose it. So I spend a few minutes every morning writing to a randomly generated prompt. So I would imagine that’s a lightweight thing.

You mentioned the update that I send to the team. This is calling on our CRM data, and that I would imagine is sort of a medium because that’s only one piece of software. But once a month, I’m calling on our CRM and our financial data and a couple of other sources, so that would be a heavy task. So on a day-to-day basis, the scheduled tasks that I have are fairly lightweight. But then when I get into the real thinking, that’s when—so I was working on something this morning on behalf of the team. I was engaging a plug-in, I was engaging the Google Drive connector, I was engaging the Google Search connector, I was engaging that deep thinking of “put all this information together,” and all of the skills that are involved: the skill of building a Word doc, the skill of building a PDF, the skill of building an HTML interactive page, the skill of building a PowerPoint—all of those in one specific task. So I would say that is a heavy task, even though it looks at the surface like a lightweight task.

Christopher S. Penn: I would say, and I think this is a fair characterization, you probably do two heavy projects a day in Cowork because you’re constantly doing deep strategy and things. So I’m going to put two a day—this is a monthly calculus—put down 60 there. Now for Kelsey, I would say Kelsey at least does at least one light and one medium task in Claude per day. I think it’s actually more than that, but I’m going to put that down as a starting point. What do you think?

Katie Robbert: I think that’s a fair starting point.

Christopher S. Penn: Okay. For me, I work in Claude code, which is slightly different, but since we’re just trying to get a sense of what Cowork will cost, I’m going to do the equivalent. On a day-to-day basis, I probably do five tasks that are light, so that’s going to be 150 of those a month. I probably do 10 tasks that are medium, so that’s going to be 300 a month. And I probably—actually, I know I do over 10 tasks a day that are heavy, that are like pure heavy code lifting. So that’s going to be another 300 there for John. John really doesn’t use Claude much at all, I don’t think. So maybe like 30 at most.

Katie Robbert: Yeah, I think so. We have a skill that was built specifically with his role in mind, and he runs it maybe once every couple of weeks. When I look at the weekly tasks—so this is looking at a month at a glance—I would actually bump up the medium tasks for me because I have weekly reports that are run that engagement, the Claude Chrome extension, the connections to our CRM, connections to our project management software. I have eight of those weekly.

Christopher S. Penn: Okay, so you’re basically running two mediums a day. Effectively.

Katie Robbert: Yeah.

Christopher S. Penn: Claude or Microsoft Copilot Cowork bills on what are called credits because why make this easy? Light tasks bill 125 credits, medium tasks bill 500 credits, and heavy tasks bill 1,200 credits. The cost is a penny per credit. So our Microsoft Copilot Cowork cost—are you ready for this, Katie? $1,600 a month.

Katie Robbert: Get out. We’re going back to candlelight and whittling pencils.

Christopher S. Penn: That is because it’s a penny per credit, which they do to make it sound cheap, not realizing that a single heavy task is 1,200 credits. So a single task is $12. So for me to do one QA run on a piece of software is swipe the credit card for $12. On a monthly basis, we are consuming effectively 657,000 credits, which is $6,570 total, all in. It’s $1,600 per user. So Katie, our Trust Insights Copilot Cowork bill is $6,570.

Katie Robbert: I have no words. That is insane. And to be fair, so you and I, Chris, I would say are power users. We are turning to these tools to do all kinds of things all day long. Even with trying to do things and schedule them off-hours to not be during peak usage, we’re still using up usage. And yeah, we are a small team. If we take out the work that Kelsey does just for the sake of this example, you and I are still eating up the majority of the cost. If we take out you, I’m still eating up a majority of the cost. I don’t know how a company or team is supposed to be able to afford to use this. It’s a real bait and switch. Shame on Microsoft.

Christopher S. Penn: Well, this is enterprise. They can do this.

Katie Robbert: Yeah, they can. It doesn’t mean they should.

Christopher S. Penn: So your usage, because a credit is a penny, your usage of Copilot Cowork a month would be $1,057.50. That is how much you consume in equivalent credits in the system. Now granted, we pay for the four of us to share a Claude Max 20 account; we pay $200 a month for it. This at the enterprise level, you’re talking four people, $1,600 for four people, one of whom barely will use it. Realistically, like you said, we’re probably going to average $3,000 an employee is what it will cost to use Cowork.

Katie Robbert: Which is an insane amount. For some companies that don’t even blink at that, but that’s a very small handful of companies who would feel that way about $3,000 a month. One of the things that we’re doing with a lot of our clients right now is trying to help them find cost savings in their tech stack—like how many tools can they reduce or licenses they can let go of and replace with things like Claude Code or Claude Cowork. But if they’re like, “Yeah, I want to do that exercise,” and what I have is Microsoft Cowork, I would say, “Cool, we’re not doing that exercise until Microsoft changes the billing,” because it’s going to cost you 10x more than it’s costing you now. It’s not worth it. Which is a real shame because Microsoft users have been waiting for this kind of functionality.

Christopher S. Penn: And so what I wanted to talk about on today’s podcast episode, now that we’ve worked out that this thing is going to cost you three grand a month—because one of the things that people have pointed out on LinkedIn is, “Oh great, you fired all these people so you can switch to AI; now AI is going to cost you more than the people did”—is how do we reduce AI costs? How do we use AI more efficiently? Because this is clearly a lot of money.

Katie Robbert: If only we had a few things to start with. I’m going to shock and dazzle everyone and say, “Guess what? Start with the 5P framework by Trust Insights.” You can learn more about it at TrustInsights.ai/5P-framework. At a high level, the five Ps are: Purpose—what the heck are you doing? People—who the heck’s involved? Process—how do you do the thing? (These are your SOPs). Platform—what tools are you using? (Not just the AI, but also your external data sources). And Performance—did you do the thing?

It sounds really straightforward because it is. However, a lot of people go straight to pushing the buttons and “vibe coding” and, “Hey, build a thing.” “What do you want it to be?” “I don’t know, you pick.” Without doing this work up front, yeah, you’re going to find yourself at $650,000 a month very quickly. There is no tool that allows you to skip over good planning upfront, good governance up front. Microsoft Cowork is no different from any other large language model in that you still need to have good requirements, you still need to have good prompting, you still need to have good governance, even if you’re just using it internally on your own systems. Enterprise companies, any company, has sensitive data somewhere within their SharePoint stack, within their databases, their document repositories. You don’t want to accidentally or carelessly give a large language model access to that because you didn’t plan ahead. So that’s my soapbox. I’m coming down off of it. Chris, what would you add to how to make AI efficient?

Christopher S. Penn: So planning, yes, 100% is going to make the most of the tools you have. The other question is, given these outlandish costs, is Microsoft the right system for you to use? Because Claude in Anthropic’s enterprise level is just as expensive. Companies have recently seen their burn through their entire Claude usage for the year, their budget in weeks. I think it’s Uber that burned their 12-month budget in a month and a half in terms of their token budget. So when we look at these prices, Katie, you remember a while back I had said, “Hey, Nvidia’s got this cool little desktop box. It’s $5,000.” You’re like, “You’re not buying $5,000 worth of hardware.” Absolutely not. Now if Microsoft or Anthropic said, “Hey Katie, you need to pay us $6,500 a month,” you’d be like, “You know what, Chris, go and buy one of those boxes; let’s buy one for each of the team and we’re going to drop Anthropic because we are not paying $6,500 a month for AI.” Right?

Katie Robbert: You know, and so it’s an interesting question because where we started the conversation was saying there’s a reason why people are wedded to using Microsoft because of the security and privacy. I don’t know that introducing an Nvidia box would comply with the regulations set forth by that company. I mean, that’s a big question. It’s an interesting workaround, but it’s not going to work for everybody, especially the more regulated the industry gets. It just might not be an option.

Christopher S. Penn: Yeah, it’s going to very heavily depend on IT. However, because it lives literally in your infrastructure, you do have a lot more governance over it because it’s literally a box that sits on your desk that you control. But more importantly, today’s top local models match a lot of the cloud foundation models and capabilities. GPU AI’s new GLM 5.2 matches Claude Opus 4.8 capabilities. Now you’re going to need a few of those Nvidia boxes to be able to load and run it well for a small cluster of employees. But for the lighter models like Qwen 3.6 or Google’s Gemma 4 if you have to, or Nvidia’s Neotron Ultra if you have to use a US-based model because of regulatory reasons—like you’re not allowed to use anything Chinese, regardless of the fact that it’s on your infrastructure—those are options that you would then use a tool like Open Cowork to handle the inference for it.

So my suggestion is that to Katie’s point, use the 5Ps and then drill down and say, “What are the things that we absolutely positively have to use Cowork for?” Or can we make that task as deterministic as possible using command-line tools and stuff that do not require AI? So for example, Katie, when you query HubSpot every day with Claude Cowork, that is using the MCP connector that uses a ton of tokens back and forth. Now we don’t see it because we’re on an individual plan. The moment we’re forced to switch to a team or an enterprise plan, we will say, “Okay, we’re going to use the HubSpot command-line tool which can fetch data in and out.” And then the AI just says, “Hey tool, give me the thing,” and it goes off and does the back and forth and brings the data back and hands it to the AI. That will dramatically cut the amount of AI usage you have because a non-AI tool is getting data for you.

Katie Robbert: As you’re describing it, I want to sort of make sure I understand because you’re making it sound like it’s an easy switch from the process that I currently have built in Cowork to, “Okay, just use a command-line tool.” I’m not someone who’s well-versed in command-line tools. You’re someone who is. However, you have your own set of things to do right now. So it’s time. It’s internal resources to make those switches to make the cost savings. I just want to be clear about that; it’s not a, “Oh well, in order to save money, let me just go ahead and use a command-line tool.” Like you still have to set it up.

Christopher S. Penn: Yes, and corporate IT will be very busy doing that. However, corporate IT also likes us because they can then govern it. They can say, “Okay, we will ensure that this suite of 10 command-line tools is installed on every computer in the company, and there’s a joint service key that we can maintain programmatically and rotate every 30 days and stuff like that.” So that infrastructure, which corporate IT is very well-versed in, is going to be much happier with that than kind of like the whole shadow IT where people are like, “Oh, I’ll just have Claude make me this thing.” No, they would much rather say, “I would like to have control over the command-line tools that are installed on every machine in the company.”

Katie Robbert: So work that out. You’ve worked with IT teams before. How likely is it that they’re going to—if you say, “Hey, I would like to have control over the command-line tools on every machine in the company,” they’re like, “Yeah, sure, Chris, no problem. Let me bump you to the top of the list. You’re a priority now.” I think you’re going to have a hard time. Like, we see the value in it, we know that it’s a useful thing. I just want to be realistic, and I’m trying not to derail the conversation too much, but I just want to be realistic that, like, yes, that’s the thing. If you have the skills to do it and if you don’t have to go through your IT team to do it, absolutely do it. If you have to go through your IT team and they have to set it up, get comfy, get in line; you’re not a top priority right now.

Christopher S. Penn: Yeah, well, my perspective is IT would want to do that. It would be like, “We would love to have more control over this to stop the shadow IT that’s happening all over the place because of AI.” So IT in its MDM config would say, “Okay, these are the 10 tools that we’re going to drop on every machine, and we’re going to also programmatically alter your Claude MD files and stuff to tell Claude this is what’s installed. You must use it so that it cuts those costs.” And IT can then say, “We certify these 10 command-line applications are safe to use.”

Katie Robbert: Provided it has the time to get skilled up to do that. So yeah, I like to make sure that we’re very clear about caveats because in the 25 to 30 minutes we have for a podcast, we go through things like “do this, do this,” and then it’s, “Well, what do you mean? It said no.” So let’s get back to—Microsoft has started to release Cowork, their version of Cowork, and we’re talking about AI efficiencies. When you think about starting places for someone who’s using Microsoft, someone who’s using their Cowork version, what is the first thing you think somebody should do before they start burning tokens or usage or spending pennies?

Christopher S. Penn: The five Ps, the planning, and build all of your prompts and all of your infrastructure for Cowork in regular Copilot, because regular Copilot is very smart. Now in regular Copilot, if you go in the upper right-hand side, there’s a little menu, a little drop-down saying “models,” and you should choose for planning. Choose GPT 5.5, soon to be 5.6—”think deeper,” that’s the smartest model that’s available. And say—and that’s where you have your conversation like, “Oh, I want to do this in Cowork. I don’t want to do this, I want to do this. Help me figure this out. Ask me questions. Let’s plan this out. Here’s the Trust Insights 5P framework. Help me use this to come up with these plans.” So you do all of your planning and all that heavy token usage in regular Copilot to build the skills and the pieces that you can then drop into Cowork, so you don’t have to use Cowork to plan because Cowork is going to chew up your usage. That way, if you can use regular Copilot, it should be a little bit lighter on your budget.

Katie Robbert: And I think one of the questions that you should add into your planning is, “Can I do this in Copilot or do I need Cowork for this?” And you know, I want you to use your human judgment, but it would be a good idea to ask the large language model like, “Do you have the capabilities to do this within Copilot or do I need to bring this into Cowork to actually execute it?” Because you may be surprised. You know, to Chris’s point, the models are getting smarter every day. And so you may not need to execute what you think you need to execute in Cowork; you may be fine with using Copilot. Yes, I get it’s not as shiny and as exciting, but you know what’s also not exciting? Being told you owe the company $60,000. That’s not exciting.

Christopher S. Penn: Exactly. Even for something like scheduled tasks—Microsoft Copilot tasks are scheduled tasks—so if it’s not something that needs Cowork’s horsepower, that will obviously keep you from chewing up those extra credits over there.

Katie Robbert: Yeah, and I think that’s a good best practice for a lot of these tools, you know? So can you do your planning in Claude Chat before bringing it into Cowork? Can you do your planning in Gemini before bringing it into their version of whatever that is? And that’s just a good best practice for efficiency in general.

Christopher S. Penn: Yeah, I mean, when I do my planning for even software builds and stuff like that, the first thing I do is I have a master planning prompt. It’s actually a skill that incorporates the 5P framework by Trust Insights. And so I have the model ask me questions from the 5P framework: “What are you doing? Who’s it for? How should it work? What are the additional command-line tools that we should be using? What is the definition of done?” And all of that is stuff that if I don’t dictate it out loud, it knows to ask me for it. So I can plan first, and then the language model rebuilds the prompt into something that meets all of those conditions and produces a really solid output that I can then go use to build requirements documents and all the stuff. You will save so much time and money by investing more heavily in planning up front, and you can then hand off the execution of the plan to a very small, fast model.

Katie Robbert: And I think that’s a really good pro tip. And I just want to give a small plug—you can actually download, we have for sale in our academy at Academy.TrustInsights.ai, a “prompt-to-skill.” So basically, as Chris was just describing, he has a specific process for building those requirements. This prompt-to-skill will help you do that and get more efficient at building those requirements. And then what you may find that you have is a reusable template, and it makes that even more efficient. So start with that. Go to Academy.TrustInsights.ai, purchase the prompt-to-skill—it’s very awkward to say that—and then start building out those requirements before you bring it into something like Cowork. And you’re going to save yourself a lot of time and money, and people are going to be like, “Wow, you did that really fast. How did you do that?” And you’ll be like, “I don’t know, I’m just that good.” But in the back of your mind you’re like, “I use the 5P framework by Trust Insights. It got me there faster.”

Christopher S. Penn: Exactly. So Copilot Cowork from Microsoft is now generally available. Before you type one character into it, please take the time to use the 5P framework by Trust Insights. Take the time to understand what your company has budgeted. Take the time to understand what tasks fall in each category, and as best as you can, try to reserve it for the things that truly need Cowork’s capabilities. And don’t just make it the default. If you’ve got some thoughts about the new Microsoft Copilot Cowork that you want to share, pop by our free Slack group. Go to TrustInsights.ai/analytics-for-marketers where you and over 4,700 other marketers are asking and answering each other’s questions every single day. And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on instead, go to TrustInsights.ai/TI-Podcast. You can find us in all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.

00:00 – Introduction 03:15 – Defining Enterprise AI vs. SMB AI 07:45 – The role of Microsoft Copilot in regulated environments 12:20 – The 20 components of Enterprise AI readiness 18:10 – Challenges in organizational adoption and change management 22:30 – Security and data privacy as the foundation 27:00 – Call to action

Watch this episode to master the complex landscape of regulated AI and safeguard your company’s future.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights, we are talking about Enterprise AI 101. I am in the midst of a series in the Trust Insights newsletter, which you can get at TrustInsights.ai/newsletter. Part one was last week on seven different aspects of enterprise AI. But Katie, you said it would probably be helpful to level set what enterprise AI is and how it differs from SMB AI, mid-market AI, consumer AI, and so on.

Katie Robbert: It is interesting because I feel like every time we jump on to record a podcast, there is a whole new set of vocabulary that I need to get caught up with. We need to make sure that everyone else knows what we are talking about because there is nothing worse than listening to a podcast or reading an article and having no idea what the author is talking about because they are introducing a concept but not really explaining it. I wanted to take this episode to talk about what enterprise AI is. Since you and I have not defined it, I am going to take my best guess at what enterprise AI is using some logic and deduction.

I could be wrong, and that is why I think it is worth covering. From my perspective, if I had to put a definition to it, I am assuming enterprise AI is the type of AI implementation that occurs at an enterprise-size company. That sounds overly simplistic, but the bigger the organization, the more red tape, the more politics, the more departments, the more stakeholders, and the more governance there is. There are a lot more complications versus a small business like we are, where we can just decide one day, “Hey, I am going to start using this tool.” There are no real hurd]]> https://www.trustinsights.ai/wp-content/uploads/2026/05/In-Ear-Insights-Enterprise-AI-101.png In-Ear Insights: Enterprise AI 101 false no 0:00 No no https://www.trustinsights.ai/blog/2026/05/in-ear-insights-setting-up-agentic-ai-for-success-part-1-job-descriptions/?pk_campaign=feed&pk_kwd=in-ear-insights-setting-up-agentic-ai-for-success-part-1-job-descriptions Wed, 06 May 2026 10:30:07 +0000 0aae3374-0209-5ef0-b693-524207af01dd In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss setting up agentic AI systems by fixing your foundational documentation. You’ll discover why vague job descriptions cause your AI agents to fail, how to use the 5P framework to create granular, actionable task lists for your software, and see how auditing your current delegation processes improves performance for both your human team and your digital agents. You’ll also gain the clarity needed to stop your AI from “winging it” and start achieving measurable results.

00:00 – Introduction 03:15 – Why most AI agents fail 07:40 – The 5P framework for AI 12:20 – Why specificity matters for models 18:50 – Auditing tasks with the TRIPS framework 22:15 – Call to action

Watch this episode to master the art of delegating to AI and become a more effective manager.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

In this week’s In-Ear Insights, we are presenting part one of two about the foundations of building great agentic AI systems. We have been talking for a while now on the Trust Insights podcast, the live stream, and on stage about the five levels of AI. Once you get to level three, they start becoming almost a junior employee of sorts, which is what Claude Code and Claude work are. Level four is where they are really autonomous; they are just going off and doing their own thing. Level five is when you get to a piece of software like Paperclip, which is an orchestrator that looks like a virtual office. It is really kind of creepy in some ways.

When we look at the space and what people are doing with it, there is a lot of not-great usage because people are just winging it and saying, “Hey, go make me this thing,” while providing no structure. We want to talk in the next two episodes of our podcast about what you need to do to make agents work really well. Katie, this is where I am going to look to you, because this is not my forte. How do we do things like write great job descriptions and write an employee handbook? If we are going to create a virtual organization, you probably need them. Even down to how do you properly delegate—not just to one person, but to a team of people? Let’s start with the job description itself. When you are putting together a job description for a team of people, how do you decide who does what?

That is a great question. I would typically start with something like the 5P framework. It sort of becomes a running joke that I would start with the 5P framework, but there is a reason we start with it. We start with it b]]> https://www.trustinsights.ai/wp-content/uploads/2026/05/In-Ear-Insights-Setting-Up-Agentic-AI-for-Success-Part-1-Job-descriptions.png In-Ear Insights: Setting up Agentic AI For Success Part 1, Job Descriptions false no 0:00 No no https://www.trustinsights.ai/blog/2026/04/in-ear-insights-updating-mental-models-and-old-knowledge/?pk_campaign=feed&pk_kwd=in-ear-insights-updating-mental-models-and-old-knowledge Wed, 15 Apr 2026 09:24:32 +0000 18b2ae8f-c7c5-53b9-9d86-0a9cf8034f60 In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss how you can keep your professional knowledge relevant despite rapid shifts in technology and software. You’ll discover how to leverage agentic AI to audit and modernize your outdated standard operating procedures. You’ll learn the vital importance of maintaining human oversight to prevent the loss of critical expertise. You’ll understand why curiosity remains your most valuable asset for effective leadership in the age of automation. You’ll see how to balance the speed of machine-led updates with the necessity of human critical thinking.

00:00 – Introduction 03:15 – Why keywords matter less in the age of AI 07:45 – Using agentic AI to update old SOPs 12:20 – The risk of cognitive offloading and knowledge decay 17:50 – Maintaining human leadership and curiosity 22:10 – Call to action

Watch this episode now to learn how to stay ahead of the curve without losing your competitive edge.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In-Ear Insights, let’s talk about updating old knowledge. Katie, you’ve been doing some work on updating standard operating procedures about Google Analytics. I’ve been putting together slides and workshops for SEO and PPC professionals about the way things are. One of the things that I noticed, particularly when I was digging through Reddit data, is how much focus there is on things that are no longer relevant.

I’ll give you a simple example. In SEO, we talked a lot about keywords—keyword lists, keyword topics, related keywords, and stuff. There is still some marginal value to that. But with the way that things like AI mode and AI overviews operate today, and the way language models like ChatGPT operate, the keyword is essentially irrelevant as a thing to focus on. It’s not where you should put your effort. Instead, you should be putting your effort on the semantic space of a topic, which again, is not necessarily all that new. When I look at the top questions in Reddit about SEO, people are still fixated on this thing that really hasn’t mattered in about 5 years.

So, when you were doing your Google Analytics stuff, I’d love you to talk through what you’re doing on that front, because there’s a lot of stuff that we thought we knew about Google Analytics that, thanks to Google’s never-ending UI changes, is completely different. Talk to what you’ve been doing and what old knowledge you’ve had to replace.

Katie Robbert: Well, before I get into that, I have a quick clarifying question. Key]]> https://www.trustinsights.ai/wp-content/uploads/2026/04/In-Ear-Insights-Updating-Mental-Models-and-Old-Knowledge.png In-Ear Insights: Updating Mental Models and Old Knowledge false no 0:00 No no https://www.trustinsights.ai/blog/2026/04/in-ear-insights-ai-and-the-future-of-work-in-2026/?pk_campaign=feed&pk_kwd=in-ear-insights-ai-and-the-future-of-work-in-2026 Wed, 08 Apr 2026 09:12:06 +0000 010da782-23b1-532a-a99f-88230a1075ae In this week’s In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss the future of work in the agentic AI world. You will discover how artificial intelligence will impact your career. You will explore the hidden reasons behind the upcoming leadership crisis. You will learn actionable strategies to protect your job from automation. You will build essential skills to succeed in this new era.

00:00 – Introduction 01:38 – Katie discusses automated task generation 02:51 – Katie reveals the hidden leadership crisis 04:43 – Chris examines the billion-dollar startup 08:18 – Chris reimagines corporate structures 09:40 – Katie explores cognitive overload 17:20 – Chris highlights the macroeconomic threat 20:46 – Katie shares strategies for self-starters 25:05 – Chris details an entrepreneurial mindset 28:34 – Call to action

Watch this episode to take control of your career and outsmart the algorithms.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In Ear Insights, METR says only the senior will survive. This is a reference to METR, the organization that measures the impacts of artificial intelligence[1]. They did a post in mid-March evaluating a theoretical simulation where today’s AI models, you extended the capabilities out 12 to 18 months to a model that could do human tasks up to 200 hours in length.

Christopher S. Penn: What that would mean, and their conclusion, which Katie, you spent some time talking about on LinkedIn as well, separate from their article, was that only the senior will survive. Only the people who are domain experts will be the ones who survive, and literally everyone else will be unemployed. We’ve also seen this in economic data.

Christopher S. Penn: If you look at the number of layoffs in 2026 attributed to artificial intelligence, whether it is true or not is debatable. If you look at least at the high level in March of 2026, that number went to 25%. A lot of tech companies doing layoffs, which is where that comes from. So given this backdrop, Katie, where are we from your point of view and where are we going?

Katie Ro]]> https://www.trustinsights.ai/wp-content/uploads/2026/04/In-Ear-Insights-AI-And-the-Future-of-Work-in-2026.png In-Ear Insights: AI And the Future of Work in 2026 false no 0:00 No no https://www.trustinsights.ai/blog/2026/03/in-ear-insights-virtual-versions-digital-twins-and-ai-clones/?pk_campaign=feed&pk_kwd=in-ear-insights-virtual-versions-digital-twins-and-ai-clones Wed, 25 Mar 2026 06:32:36 +0000 f8d3d106-28e0-585e-a06a-48abf15dba2d In this week’s In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss virtual versions, digital twins, and AI clones. You will uncover the process of building an artificial intelligence digital twin for routine tasks. You will explore the specific steps to map your unique thinking patterns into a custom prompt. You will unlock the secret to identifying the ideal duties for your virtual clone. You will master the art of preserving human relationships while your digital counterpart answers complex questions.