So What? Marketing Analytics and Insights Live
airs every Thursday at 1 pm EST.
You can watch on YouTube Live. Be sure to subscribe and follow so you never miss an episode!
In this episode, Katie Robbert, Christopher Penn, and John Wall examine how different artificial intelligence tools structure written language.
Understanding distinct AI writing styles will give you total command over your generated text. By evaluating structural features like sentence length and word choices, you will spot the mathematical patterns behind different AI writing styles. This knowledge will transform your editing routine into a simple process that preserves your genuine human voice. Selecting the right model for your personal tone will shield your work from generic text.
Watch the video here:
So What? AI Podcast Editing: Level Up Your Workflow
Can’t see anything? Watch it on YouTube here.
In this episode you’ll learn:
- What AI writing is and isn’t
- Why “don’t write like AI” is completely ineffective
- How to understand what your specific system writes like
Transcript:
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Katie Robbert – 00:30
Well, hey everyone. Happy Thursday. Welcome to So What, the Marketing Analytics and Insights live show. I’m Katie, joined by Chris and John. Howdy, fellows. Hello.
This week we are talking about examining AI writing styles of different AI systems. It’s one of the things I said to Chris because, Chris, you’ve been sort of sharing with me and educating me about the tells of what each system has, and now I can’t unsee it. Once it becomes apparent, it becomes very apparent. So we’re actually doing a little bit of the reverse today. We’re going to go over a little bit of what the different systems’ default writing styles are, but we’re also going to start to almost reverse engineer it and see.
Katie Robbert – 01:24
I think, Chris, what we’re doing today is we’re going to see which system writes the most like any given human. I think that human is me.
Christopher Penn – 01:32
You are correct. That human is you. But it’s not present-day Katie; it is 2022 Katie.
Katie Robbert – 01:41
When I think back to my writing, it was a little less polished. Here’s the thing: we all start somewhere. Prior to this role, I was never asked to write on a regular basis and give my opinion on anything. That was a newer thing for me.
I remember when we first started the newsletter, Chris, I didn’t have a voice in it at all. That wasn’t because I didn’t deserve one—it’s because I wasn’t ready. I didn’t feel ready to be doing that much sharing and thinking. Now I’m definitely comfortable with it. I would imagine that in 2022, prior to any kind of generative AI-assisted tools or rewriting, John was editing my blogs for me.
John Wall – 02:32
Yeah, we were doing a lot of getting the voice tighter. It’s funny. That’s making me laugh. Thinking about 2022, I was probably a better writer then. I think I’ve gotten weak and rusty over the years.
Katie Robbert – 02:48
So where should we get started, Chris?
Christopher Penn – 02:50
Well, let’s talk about writing styles first. As a completely shameless plug, Chris and I are hitting buttons—
Katie Robbert – 02:59
At the same time.
Christopher Penn – 03:00
Exactly. All hitting buttons at the same time.
A lot of the technologies from today’s show are in our new AI for Writers course, which you can find at TrustInsights.ai/aiforwriters. One of those things in the course is a 16-page glossary of all the different nerd speak. Some of the things that AI does, for example, is anaphora. Anaphora is defined as repeating the same word or phrase at the start of two or more sentences in a row, like: ‘We build, we ship, we iterate.’
John Wall – 03:32
Right?
Christopher Penn – 03:32
That’s an example of anaphora. There are about seven pages of things that AI tends to overuse because it doesn’t have an understanding of style. It understands probability. It says, ‘Hey, these are all the things that great human writers do.’ Like an amateur on their first day on the job at an Avid video editing station wondering what all the transitions do, you see that amateur video with every single transition overused and think, ‘No one can watch this, man.’
Katie Robbert – 04:05
And star wipe. And star wipe. And star wipe. Yep, and then it swirls.
Christopher Penn – 04:12
Exactly.
Katie Robbert – 04:13
The other thing is that it doesn’t understand style, and it also doesn’t understand frequency. If it finds that something works, it just keeps doing it over and over again. That is something I find to be incredibly annoying, even if I’m looking at something prepared for internal use—not even a public-facing piece. I look at it and think, ‘I can’t. This is terrible.’
Christopher Penn – 04:41
Part of understanding AI writing is knowing what those tells are and how much models overuse them. The second part is understanding how you can measure writing, because writing is as much science as it is art. When we work with machines, they don’t understand the art because they don’t have emotions or lived experiences. But they do understand science and mathematics if we give them literal Python code to analyze it.
For example, take the adjacent pronoun opener rate: how many times do you have subject pronouns adjacent to each other? If you’re writing fiction and start every sentence across five paragraphs with ‘she’—like ‘She did this’ and ‘She did that’—
Christopher Penn – 05:31
that is a metric you can measure in text to see if the rate is low or high. Typically, when measuring text metrics, you measure frequency per thousand words. If you have 1,500 words, you can measure how many times a person uses a specific construction. You use these metrics to evaluate a piece of writing and understand that style in a quantifiable way.
None of these metrics are inherently good or bad. They are measures you apply to create a mathematical fingerprint of someone’s writing, like understanding how often someone uses contractions. John and I write very differently. Katie, you and I definitely write very differently, so these numbers will be distinct.
Katie Robbert – 06:25
Side note: what is up with Claude not using pronouns at all to start a sentence? For example, if I write, ‘We did it,’ it starts with ‘we.’ Claude will just write ‘Did it,’ and that’s the whole sentence. What is up with that?
Christopher Penn – 06:44
It depends on the context of the task, but it also depends on the model. Even within the Claude family, models write differently. They have different training datasets from different periods of time and are post-trained on different tasks.
A lot of people have noticed that AI models over the last year have unguidedly become worse writers. They’re less creative and less interesting to read. That’s because they’re being trained more for agentic use. Agents don’t need flowery prose; agents need punchy, task-oriented execution like ‘Do this, do that. Did this, did that.’
Christopher Penn – 07:30
You’re seeing the heritage of making the tool do things rather than making the tool think.
Katie Robbert – 07:40
That makes sense. On the upside, that’s good news because despite having an AI for Writers course, we aren’t teaching you how to use AI to write your content. We don’t want you using AI to write your material. We want you to use AI to edit smartly and ensure your writing doesn’t get clocked as AI slop.
Christopher Penn – 08:12
What we have in the course is a Python script called Writing Fingerprint that takes the mathematics and creates a fingerprint from a text sample of at least 500 words.
Let’s look at the sample text I started with from a 2022 issue of Inbox Insights: ‘Getting Started with a Marketing Mix Model,’ written by Katie well before ChatGPT. It contains the 5P Framework, which has been around since late 2020. It was an informative issue that helped people understand how to think about marketing mix models.
Christopher Penn – 08:58
We wanted to analyze what Katie’s writing style looks like.
For this 1,500-word piece, the closest AI model match to her style is Moonshot AI’s Kimi K3, a Chinese model. When you look at the style measures by family, Katie’s median sentence length stands out. A major issue with writing style analysis is that people rely on averages. Averages are terrible for measuring writing because creative writers have wildly varied sentence lengths and word usages.
Christopher Penn – 09:46
You want to use medians at a minimum, though interquartile ranges are better. John, you look puzzled.
John Wall – 09:53
No, I’m just thinking about how sentence size and structure are affected by story context. There are points where you are descriptive, and other points where things get active and require tighter pacing. Averaging strips all of that nuance away. It’s a classic statistical pitfall where averages obscure the qualities of the project.
Christopher Penn – 10:27
Yep. Looking at the terms in the glossary, Katie’s use of em-dashes in 2022 was zero per thousand words. It wasn’t a punctuation mark she used. Her use of voice contractions was very high. Her median dependency distance was 2.86, meaning the word and its modifier were very close together. AI tends not to do that; AI often has a large median dependency distance with lots of filler before reaching the point.
Christopher Penn – 11:11
Looking at the models Katie writes most like—
Katie Robbert – 11:12
I want to add something to that. I do use em-dashes now. I’ve had no formal writing training, so like my typing, I’m self-taught. I didn’t know what an em-dash was or how to use it correctly until our friend Anne Handley explained its purpose and usage. Between 2022 and now, I’ve learned a lot about writing. I use em-dashes now, but sparingly, because they shouldn’t drive the whole narrative.
Katie Robbert – 11:55
I probably use parentheses in the middle of a sentence to add extra context more than em-dashes, because as Gen X, over-explaining is what we do.
Christopher Penn – 12:07
We have asides for everything.
To test how different models write, we conduct an apples-to-apples comparison using a blinded method. The models cannot see each other’s work or Katie’s original text. The prompt asks the model to write a 1,500-word newsletter article on getting started with marketing mix models, covering the 5P Framework (Purpose, People, Process, Platform, Performance), an introduction, and a conclusion.
The article must be within 50 words of the target length, verified using the system’s word count tool, and formatted in Markdown. The tone must be authentic and approachable for a senior marketing manager or director. No web search or outside document tools are permitted to prevent the model from finding Katie’s original piece.
Christopher Penn – 13:40
I ran this test across every available model: Claude Opus, Claude Sonnet, Microsoft Copilot, DeepSeek Pro, DeepSeek Flash, Gemini Flash, GLM 5.3, GPT-5.6, Kimi K3, and MiniMax. Each produced its own version.
Looking at Gemini’s output, it opens: ‘If you’ve spent recent quarters reviewing your marketing dashboards with growing skepticism, you are not alone. Between Apple’s App Tracking Transparency framework, the steady phase-out of third-party cookies, and expanding global privacy laws…’
Christopher Penn – 14:27
When you examine these writing styles, they are distinct. People often refer generically to ‘AI writing,’ but every model possesses its own unique writing style.
Katie Robbert – 14:45
That’s because every model was trained on different sets of human-written documents. Based on that data, each model determines its optimal output, producing a different result every time.
Christopher Penn – 15:09
Looking at Claude, it opens: ‘You’ve probably had a version of this conversation. Someone asks what your paid social media spend actually delivered. You pull up the dashboard and report the number. Then someone asks why that number doesn’t match the analytics tool or finance’s review. The room quiets, and the meeting moves on.’
You can see Claude’s triadic rhythm and bicolon structure: ‘That gap is not a reporting failure; it is the predictable result of a measurement approach…’ It consistently uses the ‘not this, but that’ structure.
John Wall – 15:41
You can see it right there.
Christopher Penn – 15:42
As Anthropic has made inroads into enterprise business, more online content reads exactly like Claude.
Katie Robbert – 15:57
Certain phrases give Claude away, like ‘the shape of.’ As in, ‘The shape of this is your next move.’
Christopher Penn – 16:06
Or ‘Things land.’
Katie Robbert – 16:07
Why does everything suddenly have to have a shape?
Christopher Penn – 16:14
Claude has a distinct flavor. Looking at ChatGPT-5.6, its writing is better than previous GPT-5 iterations, though less creative than the GPT-4 family. It includes phrases like: ‘Begin with the decision, not the model,’ showing bicolon frequency.
Our next step is fingerprinting each model.
Christopher Penn – 17:12
GPT-5.6 has shorter sentences, higher lexical diversity, a slightly higher median dependency distance, significant repetition, and heavy punctuation. Comparing it side-by-side with Katie’s writing shows stark differences: Katie uses frequent contractions and first-person perspective, whereas GPT uses none.
Katie Robbert – 18:00
Bizarre.
Christopher Penn – 18:00
We fingerprint every model to identify which one writes most like Katie.
The second component in the course is a YAML file containing style rules. For example, negative parallelism should not be overused—limit it to once per thousand words rather than every paragraph. When evaluating GPT-5.6 against this checklist, it scored 62. The diagnostic noted good vocabulary but poor structural variety and overused constructions.
Christopher Penn – 18:56
Its voice match was good, but its stance was off.
To conduct these audits, we use Alibaba Qwen as a neutral referee model. A model cannot audit its own writing due to inherent bias toward its own style.
Katie Robbert – 19:27
It’s the same principle as peer development: software engineers shouldn’t test their own code, and writers shouldn’t edit their own work. You need a neutral third party. When I review my own writing, I don’t see the issues until someone like John points out missing punctuation or spelling errors.
John Wall – 20:00
That’s human blindness. We go over our own work so many times that errors become invisible.
Katie Robbert – 20:08
Models experience model blindness, where they can’t evaluate their own output clearly.
Christopher Penn – 20:17
Exactly. They favor their own linguistic patterns.
A flagged item isn’t inherently bad. When evaluating Katie’s human writing against the standard, she scored 71. The audit flagged single-sentence paragraphs, load-bearing claims without named sources, and uniform staccato pacing. Flags indicate frequency awareness; moderate use of these elements is fine, but overuse becomes problematic.
Christopher Penn – 21:14
No author or model scores perfectly, and a higher score isn’t strictly better. However, this analysis quantifies human writing styles and highlights machine influence.
Katie Robbert – 21:43
Thank goodness.
Christopher Penn – 21:46
By converting fingerprint metrics and human writing evaluations into numerical scores, we perform similarity matching.
Stylistically, the model that writes closest to Katie is Moonshot AI’s Kimi K3. Second place is GLM 5.3 Flash, third is DeepSeek, and fourth is Claude Sonnet 3.5. Interestingly, Claude and Katie do not write similarly compared to these other models.
Katie Robbert – 22:49
I could have told you that. When I use AI for internal summaries or documentation, it’s clear I didn’t write it.
Christopher Penn – 23:10
Gemini is a clear outlier across all models, showing stark differences from everything else. That may stem from Google’s specific tuning and training methods. If you want native AI output that sounds distinct from standard AI styles, Gemini Flash is the best option.
Katie Robbert – 23:47
That’s interesting.
Christopher Penn – 23:50
Additionally, many models show high similarity to Claude Opus because competing AI companies train their models on Claude’s outputs via model distillation. MiniMax, for example, writes similarly to Claude for this reason.
Katie Robbert – 24:26
This entire breakdown is eye-opening.
John Wall – 24:32
Yes.
Christopher Penn – 24:33
Regarding quality assurance violations—or machine-like writing habits—Katie and Claude Sonnet share similar patterns, such as triadic rhythm usage. Sonnet is her closest match in style tells, followed by Opus and GLM 5.3 Flash.
Christopher Penn – 25:11
Blending these factors together, if Katie wanted a model that mirrors her style to minimize editing, Kimi K3 is the top match.
Katie Robbert – 25:31
I have no idea how to use Kimi K3, so I won’t be using it. In Claude, I primarily use Opus, and occasionally Sonnet for scheduled tasks.
John Wall – 25:56
Is Kimi a local model?
Christopher Penn – 26:00
No, it’s a cloud model accessed through DeepInfra. Running it locally would require around $50,000 in hardware.
Katie Robbert – 26:15
We won’t be running Kimi K3 locally anytime soon.
Christopher Penn – 26:20
It’s one of the more expensive open-weights models to host. But Sonnet remains much closer to your actual writing style than Opus.
Katie Robbert – 26:37
This investigation started after receiving feedback on our newsletter calling it ‘AI slop.’ I realized that depending on the sample, my human writing shares traits with certain models. Since models are trained on human text, these matches make sense. My writing style also evolves depending on the piece.
Christopher Penn – 27:20
Yes.
Katie Robbert – 27:21
Analyzing a different article from a different timeframe would yield different similarity rankings.
Christopher Penn – 27:27
If we analyzed a recent newsletter edited with Claude Opus, it would match Opus closely because Claude restructures text into probabilistic AI patterns.
To prevent over-rewriting, prompt the model: ‘Audit this document for me, list the smallest effective edits, and implement only those minimal edits.’
Christopher Penn – 28:14
That constrains the model from rewriting your original tone.
Katie Robbert – 28:26
I don’t use Claude to edit my writing. I draft and edit my work myself in Hemingway without using AI features, because writing is a muscle I don’t want to lose. I use Opus or Sonnet primarily for internal tasks, like call summaries or task lists.
Katie Robbert – 29:18
For public pieces, I occasionally use Claude to outline structural flow so my thoughts don’t wander. Then I write the piece myself in Hemingway. Does editing in Hemingway leave an ‘AI-written’ mark?
Christopher Penn – 29:56
AI detectors evaluate perplexity and burstiness. If an editing tool leads you toward high-probability phrasing, detector scores shift. AI detection isn’t a binary metric; it’s a spectrum based on word perplexity rates.
Katie Robbert – 30:36
AI generation is suitable for tactical copy like landing pages, but original thought leadership requires human direction. AI can simulate opinions, but it has no authentic perspective.
Christopher Penn – 31:14
To apply this, build a fingerprint document containing your text metrics and provide it to the model alongside execution rules. The model drafts text, analyzes its output against your baseline data, and iterates until it falls within one standard deviation of your metrics.
Christopher Penn – 32:11
This process requires more token consumption, but produces output matching your authentic style. Telling a model to ‘write in my style’ fails the 5P Framework because it lacks definitive performance metrics.
Katie Robbert – 33:17
Without explicit performance boundaries, AI will continue generating text indefinitely without knowing when a task is complete.
Christopher Penn – 33:42
Katie’s baseline data shows a median sentence length of 7.57 words with minimal variance between interquartile ranges, indicating a steady pace with low burstiness. Giving a model this 540-line diagnostic provides clear parameters to match.
Katie Robbert – 34:41
My business writing focuses on consistency and conciseness, which explains the low burstiness score. It’s interesting to see creative expression measured mathematically.
Christopher Penn – 35:19
Writing styles vary across time, mediums, and audiences. Texting a friend uses a completely different register and rhythm than writing a LinkedIn post or newsletter. Applying a single benchmark across different contexts produces poor results.
Katie Robbert – 36:10
I text using full sentences, proper grammar, and punctuation. It isn’t aggressive; it’s just standard writing mechanics.
Christopher Penn – 36:29
On mobile devices, double-spacing inserts periods automatically, which younger generations sometimes misinterpret as tone.
Katie Robbert – 36:41
My text messages match my LinkedIn writing style because I write consistently across platforms.
Christopher Penn – 37:10
Context matters. Standard corporate brand guidelines are often generic. Feeding bland brand guidelines into an AI model produces generic slop because it lacks human nuance.
Katie Robbert – 38:13
Companies often confuse social media banter with corporate voice. An unhinged tone on social channels doesn’t belong on a formal product sheet.
Christopher Penn – 38:48
The Daily Show with Jon Stewart had a completely different voice than The Daily Show with Trevor Noah, despite representing the same brand.
Katie Robbert – 39:07
John, how would you describe your writing style?
John Wall – 39:18
My writing is auditorily driven. I translate my spoken voice directly onto the page, combined with standard punctuation mechanics and New England sarcasm. AI cannot replicate a regional accent or authentic voice.
Katie Robbert – 40:03
It really can’t.
Christopher Penn – 40:05
To find your best model match, take the AI for Writers course at TrustInsights.ai/aiforwriters. Fingerprint available AI models and your own writing samples. Start your drafting with the model that naturally aligns closest to your baseline metrics to minimize editing time.
Katie Robbert – 41:07
Starting with an ill-fitting model wastes time and forces full rewrites.
Christopher Penn – 41:18
As AI transitions toward agentic execution, providing precise instructions and style data becomes essential. Naive AI outputs will decrease in stylistic quality, requiring more user direction.
Katie Robbert – 42:03
Maintain your critical thinking and writing capabilities. When using AI assistance, select the model best suited to your style to preserve audience trust.
Christopher Penn – 42:22
Subscribe to the show, check out our podcast at TrustInsights.ai/tipodcast, subscribe to our newsletter at TrustInsights.ai/newsletter, and join our free Slack community at TrustInsights.ai/analyticsformarketers. See you next time.
|
Need help with your marketing AI and analytics? |
You might also enjoy: |
|
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! |
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. |
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
Newsletter
Podcast
