So What? Marketing Analytics and Insights Live
airs every Thursday at 1 pm EST.
You can watch on YouTube Live. Be sure to subscribe and follow so you never miss an episode!
In this episode of So What? The Trust Insights weekly livestream, you’ll learn how to extract insights from qualitative YouTube data with AI. Discover how to identify patterns and categories from large bodies of text using generative AI. Understand how to apply these insights to validate your marketing strategies and improve product offerings. Learn how to leverage qualitative data with AI to make informed decisions for your business.
Watch the video here:
Can’t see anything? Watch it on YouTube here.
In this episode you’ll learn:
- How to access your YouTube Data
- Preparing your YouTube data for analysis with generative AI
- What you can’t analyze with generative AI
Transcript:
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Katie Robbert – 00:00
Well, hey, everyone. Happy Thursday. Welcome to “So What? The Marketing Analytics and Insights Live Show.” I am Katie, joined by Chris and John.
Christopher Penn – 00:36
Howdy, fellows.
Katie Robbert – 00:37
Oh, we got it. First try this week, we’re talking about analyzing your YouTube data with generative AI, and we wanted to talk specifically about YouTube because this is our anniversary. We started doing this live stream, what, five—five years ago. Oh, Chris has the—
John Wall – 00:57
He was specifically told not to launch fireworks.
Katie Robbert – 01:03
And it’s funny. I’ll tell a quick story. The reason I remember that this is the anniversary of us starting the live stream is because on the very first episode in September, my husband was taking out the air conditioner on the floor above me. Instead of pulling it in, he accidentally pushed it out and launched it. It went out the window on the second floor in front of my office window. My two dogs at the time and I, nobody heard it, nobody noticed, but the poor guy came running down the stairs all panicked, knowing that I was on a live stream and trying to signal me to make sure everyone was okay. I didn’t know what had happened. It was a whole thing. That was our very first episode, and look how far we’ve come. Guys, are you excited?
Katie Robbert – 01:49
Five years later.
Christopher Penn – 01:50
Are there any air conditioners in motion today?
Katie Robbert – 01:54
No, not today, but since we took it out on Sunday, that was what reminded me that this was about the time that we likely started the live stream. When we looked into it, we saw, yes, it was indeed the anniversary of when we started the live stream. So, congratulations, guys. Five years.
Christopher Penn – 02:12
Five years.
Katie Robbert – 02:14
I don’t see an end in sight.
Christopher Penn – 02:15
They.
Katie Robbert – 02:16
The purpose of us starting this was to really showcase how to do things. It’s really a how-to. The name “So What” is really that. When you get a report, when you get analysis, when somebody hands you a document, the first question you should be asking, respectfully, is, “So what? What’s in this for me? What do I do with this information?” That has always been the core mission of the live stream.
Christopher Penn – 02:46
Exactly. It is interesting because it originated from my—the live stream I was doing on Saturday nights where I was just messing around, and Katie was like, “This is not bad, except for the fact that you’re doing things at literally like 8:00.”
Katie Robbert – 03:06
8:00 on a Saturday night on Facebook.
Christopher Penn – 03:08
Exactly, exactly.
Katie Robbert – 03:10
I was like, there’s probably an audience. Let’s do this. We can do better.
Christopher Penn – 03:14
Yes. And a bit—a bit more focused. Then here’s what I’m messing around with today, with absolutely no strategy whatsoever.
Katie Robbert – 03:22
Yes, well, and there is still a bit of that. We just try to package it up a little bit more formally, but even that sometimes we don’t hit the mark. So with that, we thought it would be good to—because the live stream is broadcast on YouTube and other channels, but primarily YouTube, and it lives on YouTube, we thought it would be a good opportunity to do a demo of analyzing your YouTube data with generative AI. Where should we start?
Christopher Penn – 03:54
I would say probably we should start at the same place that we always start everything, literally all the time, which is the 5P framework. Why would we want to do this?
Katie Robbert – 04:05
If we’re talking about the live stream, then we would start—well, we would always start with the purpose. That’s what question are we answering? What is the problem we’re solving? The question that we’re answering with our YouTube data by looking at it is likely, what is the engagement of our “So What?” live stream? Is it doing anything? Is the audience growing? Do people care about it? Is it a show that we should continue to spend time on? Because it does take time out of our schedules. John has stats to crunch, Chris has science to data and data to science, and I just need to boss people around, so do we need to spend the time continuing to do the show?
Katie Robbert – 04:50
Is it making any kind of impact?
Christopher Penn – 04:53
Right, so there are a bunch of different ways to go at this, but one of the things that we’d want to think about is, do we even need AI to analyze the data? Which is a perfectly fair question. The answer might be yes; it might be no. It’s unclear. The second thing is that YouTube’s data is kind of hard to work with. If you’ve not worked with YouTube data before, YouTube provides in YouTube Studio this lovely thing called YouTube Analytics. YouTube Analytics comes in two flavors: advanced mode and not advanced mode. Not advanced mode is not terribly helpful. Advanced mode at least tells you a bit more about what’s going on. So if I look at 2025, I’m going to—
Christopher Penn – 05:40
I’m looking here at our channel, and I’m looking at a breakdown by content, and I’m looking at your typical metrics. What we can see just at a very cursory glance: our top three videos by views are the podcast, followed by a short, which is derived from the podcast, followed by an episode of “So What” on Vibe Marketing, and then a couple of one little silly thing, another live stream, and then some ads, and then more podcasts. Just at a very cursory glance, you can see that what gets views on our channel is mostly our podcast, which is good because I—and we’re—that’s—we’re doing that to be kind of weird if we didn’t. The live stream is at least in there somewhere, and it’s not completely absent.
Katie Robbert – 06:33
I mean, I see a bunch of entries for the live stream.
Christopher Penn – 06:36
Yeah, once you scroll down, you start seeing more of the live stream, but the podcast is definitely the premier property on YouTube. Now, this alone is useful, and if you wanted to just take a screenshot of this and feed this to Google Gemini, you could say, “Hey, give me a nicer-looking report that I could use with this to show the CEO that what we’re doing on YouTube actually works.” So let’s start with that. We’re going to take this screenshot here. We’ll go into Google’s Gemini. Use the AI tool of your choice. Claude is fine. ChatGPT is fine. Gemini is fine. Copilot, not so much because it doesn’t really have a very good canvas.
Christopher Penn – 07:19
I’m going to select Canvas. I’m going to say, “From this YouTube data, create a one-page report for the CEO of Trust Insights to explain what is working and not working on our YouTube channel.” The podcast we do is called In-Ear Insights. The live stream is called “So What.” There are YouTube shorts in the data. We will want to look at things like views, subscriber acquisition, and impression click-through rate as metrics that our CEO might be interested in to understand the performance of our YouTube channel to date. So if we put that there and we grab our screenshot, put it in here. Our CEO cares about acquiring audience and then getting that audience to do something, like head to our website or at the very least engage with our content.
Christopher Penn – 08:12
Our CEO also likes things to be very direct and clear, having good quality reporting. Build the report in the canvas using HTML, CSS, and JavaScript so that any math you do is computed in JavaScript. Build the report now in the canvas. So we’ll give it that as a jumping-off point just to report on what is in here. However, this is—I would call this the starting point because this is really—well, Katie, do you want to talk through the hierarchy of analytics? Because I know you talked about it in our new course, the AI strategy course, which you can get at Trust Insights, AIstrategy course. Do you want to talk about the hierarchy of analytics?
Katie Robbert – 08:59
Yeah. So, and this is where I think people get a little confused because you have to start at the foundation. If you think of it as building your house from foundation to roof and chimney and all that stuff, the foundational layer of your data is your quantitative analytics—the “what happened,” your numeric values. That’s where you’re starting. That’s your foundation. If you start to jump ahead to your qualitative data or any kind of future forecasting, then you’ve totally missed getting things squared away first. So start with the quantitative data—the “what happened.” Then you can move on to the qualitative data, which is your feedback data, your reviews, your comments—that’s your “why did it happen?” Because people are going to give you more information.
Katie Robbert – 09:53
I gave you a thumbs down because I don’t like the shirt you’re wearing, or I gave you a thumbs up because John has cool hair. So those kinds of things, that’s the “why.” And then you can move on to “what’s going to happen next.” That’s when you can take those data layers, the quality—I always get them confused—quantitative and qualitative, and start to think about future forecasting. That’s your predictive. And then you sort of move up from there. But don’t skip over that foundation.
Christopher Penn – 10:24
Exactly. So what Gemini spit out just from this screenshot was, “Hey, from this image, you have 5,700 views, 28 new subscribers, and an average cook period of 1.76. What’s working? The podcast high average view count of 1,453 accounts for significant share of total views, subscriber acquisitions are relatively low. The live stream has the highest weighted average impression click-through rate. So the live stream attracts new folks to the channel. And then shorts are just what they are.” So it says, “Hey, amplify the live streams. This is the most effective tool for acquiring new audience. Analyze, look through on any insights episodes, consider updating thumbnails and titles,” etc. “And don’t bother doing YouTube shorts. They don’t work for us, at least from this particular set of data.” Now, this is just one screenshot, so this is not the whole table.
Katie Robbert – 11:18
I’m not surprised that YouTube Shorts don’t work for us. When I think about YouTube, it’s a social platform, but it’s also a search engine. Taking the Shorts out of context is tough for a platform like YouTube. We can get away with the short videos on LinkedIn or Instagram because we can give a whole narrative to go along with what’s happening in the episode. But to take a short and just a snippet of a quote, I personally feel like that doesn’t totally translate on a YouTube Short. Whereas I feel like a YouTube Short is better used for someone to show a quick recipe of how something’s done. Something a little bit more engaging than just a talking head is the way that I think about YouTube Shorts. What about you, John? What’s your opinion on them?
John Wall – 12:12
Yeah, this is a constant B2B struggle. Can you do anything that’s interesting? A big part of it is just because so much of Shorts is spectacle. The same with TikTok, right? It’s just showing crazy things happening. You’re much better off with some puppies or somebody getting hit by a truck than talking about B2B stuff. It’s unfortunate because I’ve seen situations where we have a partner that we work with who has heavy machinery in their factory, and they run these videos, and they get hundreds of thousands of views, but none of it translates to business. People are just interested in watching this stuff happen, but it doesn’t come across. So, yeah, I’m right with you that your Shorts—unless you have—
John Wall – 12:57
If you’re product-driven, if you’re going to show—I bought these things that go in my car between the seat and the console. You know how you always drop your phone down there? Well, you put this thing there, and it catches your phone. In a Short, I saw that. I was like, “Oh my God, I need that.” I don’t care where it’s from, who it is, I’m buying it. But unless your product is that simple, Shorts are not going to cut it for you.
Katie Robbert – 13:18
So you’re the guy in the infomercial that they start with the “oh, no,” and it turns black and white, and then suddenly your world is coloring when you have that solution.
John Wall – 13:27
When I’m not—got my hand wedged between the seat and the console trying to get my phone back. I’m just like, “I need my phone.”
Katie Robbert – 13:37
Alright, Chris, we’ve gone off track long enough. So we talked about Shorts. So we’ve been talking about the quantitative data that comes from YouTube Analytics, and this is what Gemini has started to put together right now.
Christopher Penn – 13:53
Here’s the problem with this. It’s missing a lot of data, right? Because you can only screenshot so much, and we have a lot of content. It’s also not terribly prescriptive other than on a format basis. For example, “Shorts work less well than podcasts or live streams.” But beyond that, it’s not particularly insightful. So what are the things that you would want to derive from this kind of data?
Katie Robbert – 14:26
I would want to know if—and I’m not going to limit myself to whether we can or can’t answer this question with this data—but in doing this kind of content, I want to know if we’re actually helping people to do a thing, and are we reaching the right audiences? When somebody’s watching the video, are they going, “This is really good. These are really good insights. This is helpful. This is going to help me do my job better. This is going to help me move my career forward.” Whatever those are. I’m more concerned about people getting something from it than I am from the click-through rates or these quantitative metrics. Yes, I care about those because I know that those are indicators of the questions that I’m asking.
Katie Robbert – 15:20
And that’s where I have to start to sort of put those pieces together.
Christopher Penn – 15:25
Right. One of the things that you’d want to look at would be average percentage viewed—how much of the episode did somebody get through? Because if the answer is zero, they didn’t find it helpful. They got in, they watched two seconds, like, “Nope, I’m out. I can’t watch an Asian talking head.”
Katie Robbert – 15:47
John, leave that one alone. You and I are not touching that.
Christopher Penn – 15:50
Not in this episode.
John Wall – 15:51
Did I tell you about the thing that goes in my car?
Christopher Penn – 15:58
So what we can do, however, is we can put that percentage viewed column in here, and there’s a variety of metrics that you can look at to see, like, views, average view duration, etcetera. When it was published, et cetera. There’s a lot of different metrics in here. We could add in unique viewers, how many actual human beings watch, how many people came back, or how many regular viewers do we have? If I apply all of these metrics to this table, we start to get a different look at the universe of our YouTube channel. So we can see average percentage views. There is—oh, there’s—it says it can’t do longer than a certain period of time. So data for regular viewers is not available for March of 2025. Okay, cool. That’s fine.
Christopher Penn – 16:47
We’ll take off the regular viewers then since that data is just not available. What we might want to do is export the data. However, the challenge is YouTube only exports 500 rows at a time, which is obnoxious because it means that you can’t get all the data about all the videos if you have more than 500 of them. Because we do, we have, between live streams and podcasts and newsletters, we have like 1,700 videos. One of the things that we always want to know is something: has something popped back up that we didn’t know about previously? Maybe it got hot.
Christopher Penn – 17:31
The way to tackle that, which is a pain in the butt, but you can do it, is you have to go through and manually export a month at a time because you’ll get the 500 rows of data, of views and stuff about your channel this way. Make sure the metrics are all the same that you want to be getting. Then up in the corner here is the CSV file. So I can go down and get January. Let me go in and get February. Oops, it did not like that: 2, 28, 25, 21, 25, and export that. Then you can get the last three months pretty easily just by tapping those months there. So that’s a bit of a time saver.
Christopher Penn – 18:23
I would recommend that if you want to be doing this kind of analysis, that maybe you just go in every month and grab the data. If you do that, then you don’t have to go messing around with using the date selector. You look puzzled, Katie.
Katie Robbert – 18:38
Well, I have a couple of questions. One is, is there an API for YouTube Analytics that would allow you to, in a more automated way, get this data on a regular basis and just put it in a SQLite database or in a Google sheet or something, rather than having to do this?
Christopher Penn – 18:59
There absolutely is a YouTube Data API. To get to it, you have to register as a Google developer. You have to open a Google Cloud account, you have to enable billing on your account, then you have to enable the YouTube Data API. Then you have to download your setup a service account, download your service account key, and then plug that service account key into the analysis system of your choice. Can it be done? Absolutely. We have done that, and we’ve done that in the past. We do that for ourselves. Separately, are we doing it for today’s show? I hadn’t planned to because the hoops you have to jump through to get that working are somewhere between crazy and stupid.
Katie Robbert – 19:46
Well, no, and I don’t think we need to necessarily demonstrate that. I was just more curious. For those who have the skill set and the resources to do that kind of development. The other question I have for you is more about, let’s say—and I guess this is true for any kind of platform that has the data inside of it—could you take a screenshot of the data and almost kind of reverse engineer and say, “Here’s the data I have available. What kinds of questions can I answer with this data?”
Christopher Penn – 20:24
Could you do that? Yes, you could. You absolutely could. That’s generally not a bad idea to do in terms of practices, to say, “Well, what could I do with this information? What could I make from it?”
Katie Robbert – 20:37
I think that’s one of the things that as data analysis evolves and people are beginning their careers, they have 20 platforms in their tech stack, and they’re adding on new ones every day. It can feel overwhelming. The definitions from platform to platform are going to differ even if the labels are similar. So maybe that’s a way to—
Christopher Penn – 21:07
You.
Katie Robbert – 21:07
I think one of the things that I might do if I was just starting is I would take screenshots of the data available in all of the systems that I have to use, start to get almost like a library of “here’s what I can do with this data.” Maybe then set up like a notebook LM of data that’s available to me and questions that I can answer or where to look when somebody asks me a question, like, “Where am I going?” And kind of map it out that way.
Christopher Penn – 21:37
That’s a—I think that’s a fantastic way of doing it because it allows you to be curious, and it allows you to let the AI do what it is best at, which is that big picture knowledge that we often as humans don’t bring to the table because it is—it’s what you don’t know. What you don’t know is what it boils down to. I don’t know what I do or don’t know about a given topic because I can’t say what I don’t know. By definition, I don’t know it. But if I say, “Here’s a bunch of data,” think of it like this. This is something our account manager, Kelsey, talks about all the time. One of her primary uses for AI: open your refrigerator, take a picture of it, and say, “What can I make?” Right?
Christopher Penn – 22:23
You could do the exact same thing with your data, and you should do the exact same thing with your data.
Katie Robbert – 22:27
Yeah, I think that’s—I mean, that’s—it just sort of occurred to me as I’m looking at the data in YouTube Analytics. If I were just starting, I would probably start to approach it that way so that when someone says, “Hey, can we get the answer to this question?” I could go to my library of data points that I’ve built in like a notebook LM and say, “Where do I find answers to the following questions? What data do I have available to answer these questions to get you there faster?” Because that’s a question that comes up a lot: “Do we have the data?”
Katie Robbert – 23:02
How great would it be to be like, “Yes, we do, and here’s where it lives, and here’s the system that it’s in, and here’s the metrics that we look at to answer those questions.”
Christopher Penn – 23:11
That sounds like a data governance issue.
Katie Robbert – 23:13
Yeah, probably.
Christopher Penn – 23:15
Where could we learn more about that?
Katie Robbert – 23:18
If you wanted to learn more about data governance and how to set that up correctly, you can find more about that in the Trust Insights AI strategy course. Or if you’re looking for a more budget-friendly version, you can go to Trust Insights AI Strategic Toolkit, which takes away all of my talking points for eight hours and just gets you down to the point. In all seriousness, I think that’s maybe a topic that we should explore, and maybe try that on a future live stream and set that up. Maybe we could take a smaller asset like Marketing over Coffee. I would venture a guess, John, that you have far fewer tools in your tech stack than we do for Trust Insights.
John Wall – 24:03
Yeah, no. In fact, the principle is, the first answer is no. As far as adding new tools, we’re avoiding that at all costs. I do have to ask Chris, though, because it kind of bothers me. Why wouldn’t YouTube just dump into BigQuery? That’s right there, isn’t it? Are they trying to just force you to use their interface? What does Mr. Beast do?
Christopher Penn – 24:23
Beast do?
John Wall – 24:24
I can’t imagine that he’s pouring through this interface here.
Christopher Penn – 24:26
No. Mr. Beast would probably use third-party tools.
John Wall – 24:29
To begin with, just dump to a third party.
Christopher Penn – 24:31
Dumped a third party.
Katie Robbert – 24:32
He probably wouldn’t do it himself, either.
Christopher Penn – 24:34
Exactly.
John Wall – 24:35
He has a team; they’re not going to be doing this.
Christopher Penn – 24:39
Exactly. A lot of this is really aimed at the small- to mid-sized creator, right? People like us, but who don’t have data analysis skills. So you just look at this, just eyeball, and go, “Okay, well, I guess this video did well,” or, “This video did not do well.” So, cool. What we could do, and the next step in this process would be, what if we took all that data and started to work with it? When you download all that data, annoyingly, all the tables have exactly the same name. So you end up with data that looks like this table, copy 22. Right?
Christopher Penn – 25:18
As is typical with Google, they always put in a total row, which completely hoses your ability to analyze it because anything you’re trying to do to sum things up, it just totally screws that up. So you have to go through and edit the files as they come in to make them ready for use with machines. Again, as you pointed out, Katie, that’s something that you absolutely could automate if you wanted to. It’s relatively straightforward to do, or in this ad hoc case, we’re just going to do it manually. So our next step would be, what? How could we build something that would allow us to analyze this data in greater capacity? The answer to that would be something like Google Colab. So I’m going to start a new.
Katie Robbert – 26:03
That was going to be my guess.
Christopher Penn – 26:04
Yes. Google Colab, for those who are unfamiliar, is data science software. It is a data science software that is a coding environment. For the last six months or so, they have added Google’s Gemini into Colab in increasing amounts so that you can now have it do all the actual typing, which is nice, and it will make stuff for you. We might want to say to this thing, “Let’s take all of this data that we have here, just drag and drop it into our prompt,” and we’re going to give it a fairly extensive prompt. Let’s say we want all the CSV files, we want to ingest them all. We tell it, “Here’s what’s in the box. It’s YouTube data. We know there are issues. Here are the known issues in these files. Here’s how we should think about analyzing it.”
Christopher Penn – 27:02
Here’s what’s in the box: videos that start with “So What” are the live stream video, “Starts with Podcast” or “In-Ear Insights” is the podcast, and so on and so forth. Let’s start analyzing this data. You’ll see after I hit go, it will start to build its own to-do list. From that to-do list, you’ll notice it then creates its own plan of action, and Python code will start appearing in the window. This will take between three and five minutes to get through the data itself, assemble the requisite pieces. You can see there’s the—it’s starting to read the files and then start producing some level of analysis. One of the challenges with Colab is that you kind of have to know what to ask for. It is not a particularly—
Christopher Penn – 27:57
It is not the same as ChatGPT, where you can say, “Hey, what about this?” Colab’s version of Gemini is optimized for coding.
Katie Robbert – 28:07
If you do know what you’re asking for, then it’s an incredibly powerful tool. One of the ways, unsurprisingly, that we teach people to use Google Colab is by starting with something like the 5P framework. So it’s a way to gather your requirements. “What am I trying to do? Who is it for? What are the steps in the process? What are the platforms that this needs to be compatible with? What is the performance? What are the outputs?” Even that’s going to start to set you up for success with using a platform like Google Colab because that’s what it’s looking for. It’s looking for those end-to-end, not the sort of shrug like, “I don’t know, maybe just give me whatever it’s wanting,” those specific details.
Katie Robbert – 28:57
Having your requirements is going to be really essential for getting that done.
Christopher Penn – 29:03
Exactly. The second half of our prompt is to say, “Hey, you’ve got all this data. Now here’s how to think about analyzing it so we know the outcome we’re after, which is views.” In this example, we want more views on our YouTube channel. You could also do average percentage viewed, to say, “Hey, I care about getting people through our show.” That would be an equally good outcome. Based on that, you would give it a second prompt, and it will then pick up where it left off and say, “Oh, okay, you want me to engineer some new features?” Like, does the week of the year or the year of the month of the year matter? Are there shows that do better or worse during a certain period of time, and it will go through.
Christopher Penn – 29:46
Then you also give it, “Hey, here’s some ideas for how to do the testing.” What kind of algorithm should you use? So the LightGBM XG Boost. All the classical machine learning stuff that we talked about on Monday’s podcast, I believe. Yeah, I’m sorry, this week’s Trust Insights podcast about AI decisioning and how you can’t leave behind classical machine learning because it still—it can still work with numbers, whereas generative AI, not so much.
Katie Robbert – 30:20
It occurs to me—and this is maybe sort of like phase two of this kind of analysis—if you wanted to make a more robust analysis with real, actionable insights, in addition to this YouTube data, I would also pair it with your web analytics. If one of my questions, if my purpose is, “I want YouTube driving people to the website or taking action to fill out a form or contact something,” I probably need to provide that data. I would say we would probably also want to grab the channel data from Google Analytics, for example. But that could be a phase two once you get all of your YouTube data squared away.
Katie Robbert – 31:05
Just thinking, what are all the pieces of data I would need to fully answer a question? Again, that’s why we always start with something like the 5P framework because that gives you the opportunity to say, “Do I have everything that I need to answer the question?” The other framework that I would throw out here is the six C’s of data quality. I don’t have a fun little label for that one, but basically, it’s covering the question being asked, “Is the data clean? Is it comprehensive?” All those good things are going to help make sure that you can get to an actionable analysis.
Christopher Penn – 31:45
Exactly. It has gone through, and it has created its assessment based on the machine learning data. However, its assessment is ugly as hell and almost useless. Yeah, right. It’s not particularly helpful. What we can do, though, is we can either have regular Gemini cleaning up, or we can feed it to a coding LLM to actually build an application that does this in the background. While we’ve had this episode going on, I had Claude take the exact same prompt, the exact same data, and try to bake a report. The report being something that we can take a look at and make it a lot less statistical and a lot more useful in terms of, “Okay, what are the things that we should be doing?”
Christopher Penn – 32:50
So it has the same feature analysis, it has the same performance trends, it has model performance, but at the top it says, “Hey, here are the things you should be doing: Increase the number of podcasts. You do 6% of your top performers, higher view counts and engagement. Aim to produce podcast content two to three times a week. Shorter, more focused content.” Cotton observation. Top performance is shorter than medians, right? Because there’s a decent number of short pieces that does have some correlation. So we’re using an algorithm called XGBoost. Gradient Boost. Boosting. Gradient boosting does something that in the data science world is super useful. It looks at the interactions of different metrics.
Christopher Penn – 33:38
So instead of looking at just one measure, like average time viewed or number of views or subscribers, it tries to essentially do almost a causal analysis to say, “How much does each measure influence other measures?” Something called feature importance. From those conclusions, it can see that while length may not be a huge driver, it does contribute to the outcome that we care about. Increased Newsletter Production Plan 2, 3 Newsletter Videos a month. Okay, we’re already doing that. Good. SEO topics show high predictive importance. So do more topics on SEO podcasts. Then low priority seasonality. There’s very little seasonality in our stuff because we’re just doing this stuff all the time.
Christopher Penn – 34:22
Unsurprisingly, the podcast, because there’s so much more of it in our data, because we’ve been doing it the longest, shows the most successful pattern, and there really isn’t a pattern in terms of a particular topic. So one of the things that we had to do was engineer out of 10 different topics, like AI change management stuff, is there a topic that, based on the show title, like, “Yeah, do more of this”? Turns out there isn’t.
Katie Robbert – 34:52
We put the videos into—we tag them with different categories. Is that data not available?
Christopher Penn – 35:01
Not in YouTube, not in this report. We would have to export that from the API.
Katie Robbert – 35:07
Gotcha. That’s one of those things that, I know that we’ve made a conscious effort to tag videos—like, “This is an AI video,” or, “This is whatever type of video.” It’s a little disheartening to know that data then can’t make it. I know it’s more for the user for when they’re searching, but to also do the analysis on it would probably be helpful.
Christopher Penn – 35:29
The data is available there in aggregate. If we look at the raw data itself, we have topics. Is AI, is analytics, is marketing, is social media, is SEO, podcast topic data, science, business strategy, tools, etc. Those topics are encoded in here based on the show titles. I didn’t pull it from that part because it’s not visible in the YouTube Analytics interface. We would have to go to the API to get that.
Katie Robbert – 35:54
My point is, yes, it’s in the API, and you’re deriving these from the titles, which we’ve done a much better job about calling things what they are and not coming up with fancy marketing jargon. But that’s still frustrating. I guess what I’m saying is, for the non-API user who’s trying to just get some analytics, the fact that you can’t just get that data to analyze is a bit disheartening.
Christopher Penn – 36:20
Right. That’s again, YouTube has made their interface for the non-technical person who frankly isn’t going to bring much in the way of analytics to the party either. They just want to know what should I do? What should I do more of? There’s other stuff in here that we could certainly put in that you can put into the system to try and broadly analyze what’s going on with your channel. In your channel analytics, you can look at your overall audience. Let’s look at the overall audience for 2025. You can see, for example, our monthly audience is about 900 people. Most of our folks are new viewers. The content that’s popular with different kinds of audiences. So regular viewers are all pod—
Christopher Penn – 37:07
Most podcast casual leaves the live stream, and then the shorts are the new viewers. So there’s a case where the shorts are getting new people who might be checking out the channel. Again, we didn’t see that in the actual data. So YouTube is just very quickly summarizing stuff, possibly in a misleading way.
John Wall – 37:31
System bundling up stuff incorrectly.
Christopher Penn – 37:33
Come on. Most people watch on their computer versus the mobile phone. So there’s a lot of high-level data here, but this doesn’t make it into the advanced analytics interface.
Katie Robbert – 37:52
Because of the way that they have it spread out, I’m hard-pressed to say, “Yeah, I know exactly what we should do next, I know exactly what we need to do.” But based on this high-level analysis, I know that we’re not doing anything erroneous, that we’re doing things that we should continue to do. One of the things we started doing more recently was recording a video version of our newsletter. Chris, you’ve been doing that for a long time, but we more recently started doing it, and it’s not zero; it’s doing something. That to me is like, “Okay, so it’s not a complete waste of time.” Honestly, it takes me about 15 minutes, maybe longer if I have to restart a few times. But it’s doing something.
Katie Robbert – 38:43
Part of what it’s doing is it’s feeding AI search, which is one of the goals. But the other is, it’s just opening up the newsletter to a wider audience and making it a little bit more accessible.
Christopher Penn – 38:57
Exactly. The last thing that’s interesting is in the Content tab. YouTube has a new tab called Inspiration, where it attempts to look at the content that gets engagements on your channel and suggest future shows for you. Very often, you need to do some maintenance on your channel first before this becomes useful. Anyone who’s run a YouTube channel for more than two minutes knows you get a lot of spam bots leaving random comments that mess with YouTube Analytics, that mess with the system recommendations. So you will get recommendations here that make no sense at all because it’s going off of what people commented on and what they said in their comments. For example, I have to check in once a quarter and prune out all those random cryptocurrency spam comments because otherwise, this all becomes ideas for content.
Christopher Penn – 39:49
Make your next video about Bitcoin. No, that’s not a thing.
Katie Robbert – 39:54
Well, I’m pretty sure I’m going to be disappointed if one of our recommendations isn’t to make an infomercial about the seat saver for John’s phone in his car.
John Wall – 40:04
I was just going to say, wait, you’ve taken out all my crypto comments.
Katie Robbert – 40:11
No, I mean, and I think that’s an incredibly helpful pro tip because I look at this, I’m like, “Well, that’s not useful to us at all. That isn’t the kind of content we’d be creating.” But it speaks to, again, the spam bots that we get, right?
Christopher Penn – 40:26
The overarching thing with AI is, AI is only as good as the data you give it. So if YouTube is being fed with crap data from your channel, that is all spam, all of its recommendations are going to suck, too.
Katie Robbert – 40:41
Yeah. Now, the big question that John, I always ask you, when is Marketing over Coffee going on YouTube?
John Wall – 40:49
Got another batch of videos just sitting in a pile in my inbox, ready to be posted. The worst part is I’m paying for the tool, too, which is insane.
Katie Robbert – 41:01
So, John, you are our end user. You are our audience.
John Wall – 41:05
Yeah, yeah, I definitely am the audience. I finally, after years and years, fixed a bunch of analytics stuff for Marketing over Coffee over the past two weeks. So at least there’s some forward motion. But yeah, there’s an infinite amount of work, unfortunately.
Christopher Penn – 41:22
It says, “Yeah, your phone—the phone wasn’t holding the phone, so they wedged the front cover in the ash.” There you go. That’s one way to do it. No special device needed. You just need a phone cover that you can wedge into the ashtray of your car.
Katie Robbert – 41:40
Well, if your car has an ashtray, that says everything about how old it is.
Christopher Penn – 41:45
Exactly.
John Wall – 41:45
All my smoking and driving regular.
Christopher Penn – 41:50
I think we’re done.
Katie Robbert – 41:52
I think we’re done here. We’re done.
Christopher Penn – 41:55
Any final thoughts on our anniversary episode?
Katie Robbert – 42:00
You know, we’re five years in. Here’s to five more years. I’m honestly excited to see where else we can take this show. I think that we do a lot of good with doing demos. The question is, what haven’t we done that would be really valuable for people? That might be a question that I put to our community, our Analytics for Marketers community, free to join. But really, what are the demos that you’re dying to see that you don’t want to have to sign up with a vendor for? Is it something that we could handle? I think that’s really the next evolution: getting even more focused in on what our ICP wants to see versus what we think might be cool.
Christopher Penn – 42:51
Exactly. So go join the Slack group. Go join the Slack group and let us know. We want to hear from you. We will see you all for the next five years of “So What?” Thanks for tuning in. Talk to you next time. Thanks for watching today. Be sure to subscribe to our show wherever you’re watching it. For more resources and to learn more, check out the Trust Insights podcast at trustinsights.com/podcast and a weekly email newsletter at trustinsights.ai/newsletter. Got questions about what you saw in today’s episode? Join our free Analytics for Marketers Slack group at trustinsights.ai/analytics-for-marketers. See you next time.
|
Need help with your marketing AI and analytics? |
You might also enjoy: |
|
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! |
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. |
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
Newsletter
Podcast
