This data was originally featured in the October 18, 2023 newsletter found here: https://www.trustinsights.ai/blog/2023/10/inbox-insights-october-18-2023-be-remarkably-human-generative-ai-and-youtube-consumption/.
Let’s take a look today at building a professional development recipe, one for digesting down a lot of disparate but related information into a single, coherent work. We’re going to use several different generative AI tools; substitute in your favorite tools if you prefer different ones than the ones I’m using.
We know that YouTube is a rich source of information about nearly any topic, and there are millions of people creating YouTube content every day. However, a lot of videos have… well, a bit of fluff to them, mine included. There are the inevitable sales pitches and self-promotional parts, which are fine in moderation.
However, it takes time to consume YouTube videos, time that we might want to spend elsewhere. So what would it look like to build a process, a system, to synthesize YouTube knowledge on a topic? Let’s step through a user story to establish what our purpose is.
As a busy marketer, I want to summarize a collection of YouTube videos to a succinct summary so that I know what the latest is on a topic without investing an inordinate amount of time.
From this, we work out our basic process:
- Find relevant YouTube videos.
- Download YouTube videos.
- Transcribe YouTube videos.
- Summarize YouTube transcripts.
I’ll provide two versions of this recipe, one that is a more manual process, and then a more automated process, in case it’s something you want to systematize.
Manual Process
Before we begin, here’s a list of tools used in the manual process:
Step 1
Let’s begin with step 1. To grab the essence of a topic on YouTube, simply search for it. Let’s say we want to know what’s going on with SEO. I’d search for something like “Google algorithm update”, restricting the results to this year or even this month:
From this, I’d grab the URLs of the top 5 or 10 videos that seem relevant and put them in a text file or spreadsheet.
Step 2
For step 2, there are any number of software packages that can download videos, as well as a number of sites that do so for free. Wave Video’s site does a good job. As an added bonus, you can use it to download straight to audio.
Step 3
For step 3, we first need to convert the videos to an audio format that transcription software will recognize. Tools like Otter.ai can deal with pretty much any format of audio. This would be my suggested tool of choice, and if you used Wave Video’s downloading tool, then you already have the audio files ready to go.
Next, we transcribe our newly-created audio files. A tool like Otter is straightforward – just drag and drop your audio files into the interface and wait for it to process them. If you’re on the paid plan, you get plenty of minutes per month to do this.
Step 4
Let’s move onto step 4. You should have a pile of text files now. For non-technical users, using the large language model of your choice – I recommend Anthropic’s Claude 2 for this task – you’ll feed it a prompt that goes like this:
You are an SEO expert. You know search engine optimization, search algorithms, Google search. The following is a transcript of a YouTube video about the latest changes in the Google algorithm in the last month. Your first task is to summarize the top 10 points made in the video, but exclude any paid promotions or advertisements for services that require payment. Exclude redundant points. Summarize the video’s key points:
Feed each transcript into the large language model of your choice and examine the outputs:
Copy and paste them to a text file.
Once you’re done, you’ll have what’s essentially a briefing sheet of the top videos on a topic:
If you like, you can even consolidate this down by asking the language model to summarize and remove duplicate points:
Automated Process
For the automated process, you’ll be using command line tools and scripting to perform the same steps as in the manual process. Here’s a list of tools used in the automated process:
Step 1
Step 1 remains the same – put all your YouTube URLs into a text or CSV file.
Step 2
For batch downloading videos, I prefer the yt-dlp software, which is free and open-source. It’s a command line script that uses Python; give it the URL of a YouTube video and it will download it.
Using this software, you can either manually download each of the URLs you grabbed in step 1, or if you’re handy with scripting, write a small bit of code to read the spreadsheet and download them all.
Step 3
For converting the downloaded video files, I recommend using the free, open-source ffmpeg utility. Batch convert them using an if-else loop in the shell environment of your choice.
Once you’ve done that, install and use whisper.cpp to batch transcribe the audio files.
My choice of transcription software for automated usage is OpenAI’s Whisper, using a version optimized for the Mac, whisper.cpp. This is a highly technical piece of software that takes a lot of elbow grease to get working, BUT the advantage is that once you have it working, you can transcribe unlimited amounts of audio because it uses your computer to do the work. You never run out of minutes, except minutes in the day.
Step 4
Use the shell scripting language of your choice to programmatically feed each transcript to the large language model of your choice using their APIs and export the data to some database format. After that, export the database text fields to a flat text file.
The remainder of step 4 is the same, in terms of processing the outputs.
Conclusion
You’re done, and you’ve created a briefing on a topic in very short order instead of having to watch several hours of video to get the same information. Additionally, you can look at each of the individual transcripts if you want to dig deeper into the video to understand what was said, or go back to the video if there were key points and perhaps visuals you need to see – but instead of having to watch all the videos, you can watch just the one that has the most relevant information.
Generative AI is good at creating content, but it really shines at summarizing content. With the surplus of content we have every single day, it’s a powerful assistant for saving us time and distilling out the most important information we need to know to do better work.
|
Need help with your marketing AI and analytics? |
You might also enjoy:
|
|
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday! |
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday. |
Newsletter
Podcast
