INBOX INSIGHTS, June 25, 2025: When AI-First Goes Wrong Part 5, Why AI Can’t Do Math - Trust Insights Marketing Analytics Consulting

INBOX INSIGHTS: When AI-First Goes Wrong Part 5, Why AI Can’t Do Math (2025-06-25) :: View in browser

👉 Download our new, free AI-Ready Marketing Strategy Kit!

INBOX INSIGHTS: When AI-First Goes Wrong Part 5, Why AI Can't Do Math (2025-06-25)

Watch this video on YouTube

The Organizational Behaviors That Separate AI Success Stories from AI Disasters

If you’re just joining us, catch up on our complete analysis at trustinsights.ai/newsletter. This week: putting it all together.

After analyzing Duolingo’s disaster and dozens of other AI implementations, one thing is clear: this isn’t about technology at all.

It’s about organizational behavior.

The companies that succeed with AI versus those that create PR disasters aren’t separated by technical sophistication. They’re differentiated by fundamental differences in how they approach change, communication, and human partnership.

The Behavioral Divide

Organizations that fail treat AI as a technology problem:

Cost reduction through automation
Innovation theater to impress investors
Communication emphasizing what’s changing
Measuring what’s easy to count

Organizations that succeed treat AI as an organizational capability problem:

Value creation through partnership
Systematic change management to empower people
Communication emphasizing what’s improving
Measuring what actually predicts success

The Five Behaviors That Predict AI Success

1. Purpose-Driven Implementation (Not Innovation Theater)

Failures: Announce bold AI strategies, then figure out implementation later.

Success: Start with specific problems, validate through pilots, communicate proven results.

Von Ahn’s “AI-first” announcement was pure theater. His corrected messaging about “acceleration” and “workshops” represented actual purpose, but came too late.

2. Partnership-Minded Leadership (Not Replacement-Focused)

Failures: View AI and humans as interchangeable, measure success by eliminated roles.

Success: Design workflows where both make each other better, measure enhanced capability.

Our SWOT analysis revealed what replacement costs: institutional knowledge, innovation capacity, crisis resilience. Smart organizations preserve these while adding AI capabilities.

3. Trust-Building Communication (Not Fear-Inducing Announcements)

Failures: Sound exciting to investors but threatening to employees, then backtrack.

Success: Use ADKAR and Golden Circle principles to communicate opportunity with concrete support.

Duolingo violated every change management principle. Trust, once broken, is nearly impossible to rebuild.

4. Systems Thinking (Not Announcement-Driven Strategy)

Failures: Use public announcements to force internal alignment.

Success: Achieve consensus internally first, then communicate from proven results.

The best implementations follow systematic approaches like our 12-month roadmap, building capability quarter by quarter.

5. Human-Centered Measurement (Not Just Efficiency Metrics)

Failures: Track automation rates while ignoring trust and innovation capacity.

Success: Measure engagement, quality, and capability enhancement alongside efficiency.

The Alternative Duolingo Story

Imagine if von Ahn had demonstrated these behaviors:

Instead of: “Going AI-first and eliminating contractors” He could have said: “Giving our experts AI tools to create more authentic content faster, so they focus on cultural elements that make learning stick.”

Instead of: AI performance requirements He could have: Provided training and experimentation time to help everyone enhance their expertise.

Same AI capabilities, completely different organizational outcome.

Your Success Framework

Before any AI announcement:

Have we proven this through internal pilots?
Will people see enhancement or threat?
Do we have concrete support plans?
Are we measuring human outcomes?

During implementation, cultivate:

Partnership mindset over replacement thinking
Trust-building over innovation theater
Internal alignment before external announcements
Human-centered measurement alongside technical metrics

The Choice Every Leader Faces

Technology doesn’t determine AI success—organizational behavior does.

Every leader faces the same choice: Use AI to build capability or justify eliminating people? Invest in partnership or pursue replacement? Enhance human potential or automate it away?

Companies that choose partnership will have massive competitive advantages. Those that choose replacement will create their own disasters.

Your Next Step

You have everything needed to build success:

Communication frameworks that build trust
Analysis tools to spot dysfunction early
Assessment methods for what to preserve vs. enhance
Implementation roadmap for sustainable partnerships
Behavioral guidelines for strengthening your organization

The question isn’t whether AI will transform your workplace. It’s whether you’ll use that transformation to make your organization stronger and more resilient, or follow the path to social media blackouts and embarrassing backtracks.

Choose partnership.

Download our complete AI-Ready Marketing Strategy Kit for detailed frameworks that turn these insights into action.

What organizational behavior will you focus on first?

Reply to this email to tell me, or come join the conversation in our free Slack Group, Analytics for Marketers.

– Katie Robbert, CEO

Do you have a colleague or friend who needs this newsletter? Send them this link to help them get their own copy:

https://www.trustinsights.ai/newsletter

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss how to break free from the AI sophomore slump. You’ll learn why many companies stall after early AI wins. You’ll discover practical ways to evolve your AI use from simple experimentation to robust solutions. You’ll understand how to apply strategic frameworks to build integrated AI systems. You’ll gain insights on measuring your AI efforts and staying ahead in the evolving AI landscape. Watch now to make your next AI initiative a success!

Watch/listen to this episode of In-Ear Insights here »

Last time on So What? The Marketing Analytics and Insights Livestream, we looked at podcast transcript automation. Catch the episode replay here!

This week on So What?, we’ll be sharing how to create bonus content from your podcast. Are you following our YouTube channel? If not, click/tap here to follow us!

Here’s some of our content from recent days that you might have missed. If you read something and enjoy it, please share it with a friend or colleague!

Take your skills to the next level with our premium courses.

Get skilled up with an assortment of our free, on-demand classes.

You could be reaching 32k+ marketers, analysts, data scientists, and executives directly with your ad. Want to learn more? Reach out and contact us.

In this week’s Data Diaries, let’s explain some math – in particular, why generative AI tools like ChatGPT struggle with math.

At its heart, math – like addition, subtraction, multiplication, and division – isn’t like written prose. Calculation and statistics may both use numbers, but they do so very differently.

Natural language is predictive and probabilistic in nature. If I say, “I pledge allegiance to the”, most Americans would auto-complete that sentence mentally with the word “flag”. The probability of the next word being flag is nearly 100%.

Mathematical language is symbolic and deterministic in nature. If I say, “4 + 5 =”, we know that the numbers are symbols (symbolizing a quantity) and the + sign is an operator that mentally groups two things together. An AI tool might look at 4 and 5 and conclude the next most probable character is 6, which would be probability-wise correct but computationally wrong.

Imagine this. Suppose you ask AI for a brownie recipe. It’s seen millions of brownie recipes, and almost all of them are in standard size pans. It gives you a solid recipe. Then you tell it that you don’t have a 9×13 pan. In fact, the only thing you have is a 13 inch frying pan. If you ask it to recompute its ingredients based on that, it’s going to go off the rails because there are almost certainly no recipes based on a non-standard size frying pan.

A human would go, okay, 9×13 is 117 square inches. A 13 inch frying pan has a radius of 6.5 inches, so its area is 132.73 inches, so we need to adjust the recipe upwards by 13% for each ingredient.

AI will try to make adjustments based on what it’s seen, but since it hasn’t seen that much, chances are its math is going to yield inedible brownies.

What this means is that under the hood, AI is incapable of performing the actual math, the actual calculation. It can’t do it, and at least for now, with the current AI architectures, under the hood it will NEVER do math well. What software makers do behind the scenes is have AI write CODE (which is a language) that it executes and then does the math, returning the result to the language model. It’s an inelegant but effective hack.

This is why we always advise to hand finished results to tools like ChatGPT. Share the report, not the raw data. The AI tools will write good prose around the finished results because you’re then using numbers as language, like saying “Website traffic was down 41% last month”, rather than as symbols of computation.

Don’t let AI do math.

Here’s a roundup of who’s hiring, based on positions shared in the Analytics for Marketers Slack group and other communities.

Are you a member of our free Slack group, Analytics for Marketers? Join 3000+ like-minded marketers who care about data and measuring their success. Membership is free – join today. Members also receive sneak peeks of upcoming data, credible third-party studies we find and like, and much more. Join today!

Imagine a world where your marketing strategies are supercharged by the most cutting-edge technology available – Generative AI. Generative AI has the potential to save you incredible amounts of time and money, and you have the opportunity to be at the forefront. Get up to speed on using generative AI in your business in a thoughtful way with our workshop offering, Generative AI for Marketers.

Workshops: Offer the Generative AI for Marketers half and full day workshops at your company. These hands-on sessions are packed with exercises, resources and practical tips that you can implement immediately.

👉 Click/tap here to book a workshop

Where can you find Trust Insights face-to-face?

AFT, Washington DC, July 2025
AMA Pennsylvania, York, August 2025
SMPS, Denver, October 2025
MAICON, Cleveland, October 2025
MarketingProfs B2B Forum, Boston, November 2025

Going to a conference we should know about? Reach out!

Want some private training at your company? Ask us!

First and most obvious – if you want to talk to us about something specific, especially something we can help with, hit up our contact form.

Where do you spend your time online? Chances are, we’re there too, and would enjoy sharing with you. Here’s where we are – see you there?

Read our disclosures statement for more details, but we’re also compensated by our partners if you buy something through us.

Check out our new AI news roundup newsletter over on Substack! Get the AI news that matters to you – completely free.

Here’s the first issue!

Some events and partners have purchased sponsorships in this newsletter and as a result, Trust Insights receives financial compensation for promoting them. Read our full disclosures statement on our website .

Thanks for subscribing and supporting us. Let us know if you want to see something different or have any feedback for us!

Need help with your marketing AI and analytics?

Leave a Reply Cancel reply

Subscribe to our Weekly Newsletter

Pin It on Pinterest

Watch This Newsletter