INBOX INSIGHTS: When ChatGPT Goes Sideways, Content Curation Systems (9/6):: View in browser
September 2023 One Click Poll
Please click/tap on just one answer – this is our monthly survey to see how we’re doing, so please do take it each month!
How likely are you to recommend Trust Insights as a consulting firm to someone in the next 90 days?
We use this information to measure how effective our marketing is. There’s no form to fill out – tapping your answer is literally all there is to it. Thank you for your help!
When a Chat with ChatGPT Goes Sideways
My brain is empty. I have no idea what I’m going to write about this week. Literally nothing. Not one single small idea that I could tease out. Nothing. My brain is a void.
So, I turned to the resources I have available, specifically ChatGPT, to see if I could get inspired. Normally I would start off with a prompt following the RACE format. If you’re not familiar, RACE stands for Role, Action, Context, Execute.
Instead of crafting a prompt, I thought I would have a chat with ChatGPT. I cannot take credit for this. I had started with a prompt and was getting no where. As I was complaining about this in Slack, Chris reminded me that I didn’t need a prompt, I could literally have a chat. You know, the “chat” in ChatGPT.
I started by saying that I was stuck on what to write for the newsletter and that I was hoping we could have a brainstorm session. The system responded that it was happy to help and asked me for a little more context. I provided the context to the system.
Then things went sideways. I’m not good at having a casual chat.
Before I knew it, the system was providing solutions I didn’t ask for. I thought we were brainstorming. I thought we were just chatting. The system was trying to solve my problem and hurry me on my way.
When I managed software development teams, I also oversaw the transition from development to quality assurance (QA). The QA team was responsible for reading the requirements, following the steps, and testing the software. Their goal was to match how the software performed versus what the requirements described. The software often failed initial testing due to unclear or outdated instructions. This caused friction between development and QA. The QA team felt they were wasting time testing incomplete features. Development felt the code was more important than the requirements. Their standpoint was that the requirements were a guide, and that QA should be able to “figure it out.”
This is why I bring up the 5P Framework so often. As a reminder, the 5Ps are Purpose, People, Process, Platform, and Performance. While working with the previous team, the requirements mainly included high-level process details. However, they lacked a clear purpose and any performance metrics.
My interaction with ChatGPT was very similar to the friction between development and QA. My previous role taught me that without clear requirements, you won’t achieve good results. You’ll waste time, you’ll waste resources, and you’ll burn out your people. Today, I went in with no requirements. I had no plan other than to get unstuck. But I still got frustrated with the results I got, even though I didn’t really know what I was after.
Here’s the thing. You can open up a new instance of a generative AI tool and have a conversation with no purpose. There is nothing wrong with that. I went in with a loose purpose and an unrealistic outcome. When ChatGPT started offering solutions I didn’t want, I gave up.
These systems aren’t perfect. If anything, they are still in their infancy. If you’re looking to poke around and just ask questions, you don’t need a clear plan. But if you’re in a situation like me where you have a looming deadline and people waiting on you to finish your work, you should have a plan. Use the 5Ps. Even if you quickly sketch out some requirements it’s better than having no plan at all. And the more precise you need your results, the deeper you’ll want to document the requirements.
I’d started with a prompt, pivoted to a chat, and then gave up all together. My lesson here is to be clear on my expectations before I get frustrated. I should have refined my original prompt to be more clear. My goal was to get some topic ideas for the newsletter. Instead of rejecting the results due to my vague question, I should have taken my own advice and worked through the 5P Framework. Shame on me for wasting my own time.
Are you experimenting with generative AI? Reply to this email to tell me about it, or come join the conversation in our Free Slack Group, Analytics for Marketers.
– Katie Robbert, CEO
Do you have a colleague or friend who needs this newsletter? Send them this link to help them get their own copy:
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris provide a guide to generative AI for the C-Suite and discuss how CEOs should approach understanding and using large language models and AI, starting with identifying business problems first before considering technology solutions.
Last time on So What? The Marketing Analytics and Insights Livestream, we discovered the substantial biases in large language models in a revealing side-by-side comparison. Catch the episode replay here!
This Thursday at 1 PM Eastern on our weekly livestream, So What?, we’ll be inspecting gender-based responses from generative AI. Are you following our YouTube channel? If not, click/tap here to follow us!
Here’s some of our content from recent days that you might have missed. If you read something and enjoy it, please share it with a friend or colleague!
- In-Ear Insights: The C-Suite Guide to Generative AI
- ChatGPT and Job Losses
- So What? Gender Bias in Generative AI
- Using predictive forecasting to not send email
- Disclosure of AI and Protection of Copyright
- INBOX INSIGHTS, August 30, 2023: Foundations of Innovation, AI-Driven Pumpkin Spice
- In-Ear Insights: Gender Bias and Fairness in Generative AI
- Almost Timely News, September 3, 2023: The Future of Work in the Age of AI
Take your skills to the next level with our premium courses.
Get skilled up with an assortment of our free, on-demand classes.
- The Marketing Singularity: Large Language Models and the End of Marketing As You Knew It
- Powering Up Your LinkedIn Profile (For Job Hunters) 2023 Edition
- Measurement Strategies for Agencies course
- Empower Your Marketing with Private Social Media Communities
- How to Deliver Reports and Prove the ROI of your Agency
- Competitive Social Media Analytics Strategy
- How to Prove Social Media ROI
- What, Why, How: Foundations of B2B Marketing Analytics
Over the past week, we’ve received and witnessed quite a number of questions along this general theme:
“Does anyone have experience cataloging content or a tool they’ve used to automate content cataloging?”
“Does anyone here have a recommendation for a good tool to curate and summarize content?”
“We spend A LOT of time trying to figure out the right HOT topics and what we should be pushing out in reference to our services.”
The reality is that while there are a lot of tools on the market that purportedly can fulfill these exact tasks, they’re often burdened with a lot of extras we don’t need, missing some custom functionality we do need, and cost more than we’re willing to pay. So what’s an enterprising startup to do?
We built our own.
Why would we do such a thing? Was there such a gap in the market that it was necessary? The idea actually started when Katie and I worked together at our previous company, and that company was notorious for not wanting to spend a dime more than absolutely possible. Off the shelf solutions did exist, but they were prohibitively expensive, so we engineered a clunky workaround that served our needs at the time.
When we started Trust Insights, we had the time to rebuild the entire idea from scratch. One of the challenges of a resource-constrained environment is that you accumulate technical debt rapidly – as time goes on and you don’t have the time to fix things, patches and quick fixes accumulate like chewing gum under a bus stop bench. After a certain point, the system becomes a labyrinth of quick fixes that make it near impossible to maintain.
So when we re-engineered the entire thing from the ground up, how did we architect it? Fundamentally, when you build a content curation system, you’re essentially building a search engine for private use. That means you need a system that does three things – crawl, index, rank, or more generically, ingest, process, and output.
One of the mistakes I made early on in the old version that I rectified in the new version was trying to do everything in one place. A system that is highly modular is easier to maintain (with documentation) than a system that is a massive monolith, in the same way that your car engine is a collection of many different pieces and not just one magical contraption.
So what does a system like this look like? There’s an ingestion mechanism; we feed it the RSS feeds of blogs for a given industry as well as industry news sites. When we build these for clients, that’s usually the first question we have – what are the credible, non-competitive news sources? From there, we have a series of processing scripts that crawl the content, tag every article for the keywords and topics we care about, and then store all the information in a MySQL database. Finally, there are output modules that produce formatted text for our newsletters.
Now, there are definite shortcomings to the system. The biggest shortcoming is that there’s no interface to it. None. The way we access it is through a series of scripts we run, so if you’re not comfortable with a command line, it’s challenging to use. That’s a solvable problem if a customer asks us for it; with powerful tools like ChatGPT’s GPT-4 model that can write entire software platforms, we could easily create an interface. There’s just no need for it right now for our current use cases.
The system as it stands works well today, but we have plans for it. We plan to replace the keyword-based system with a large language model that can infer topics with much more accuracy (at the cost of greater compute power), helping curate ever more relevant content to share. We also plan to integrate our custom-built predictive SEO software into the system as well, to ensure that topics which audiences are searching for are being shared by us at the right time of year. If, for example, people are interested in healthcare analytics during a certain couple of weeks per year, we’d want to make sure we have great content on that topic ready to go, and with predictive analytics plus content curation, that’s achievable.
So what? Why does this matter? Off the shelf software packages can do the job, but at the cost of hard dollars plus a lack of customization for your actual use cases. Creating your own software has its own costs – soft dollar costs in maintenance and upkeep – but affords much more flexibility. In the age of generative AI and incredible models like CodeLLaMa and GPT-4, if you have an idea that doesn’t exist yet (or is out of reach financially), it’s worth trying to build your own. At the very least, you’ll have a greater appreciation for the process of software engineering, and at the most, you might just build a new business within your business. So to the folks asking the questions at the beginning of this piece, give this a try. You might surprise yourself!
- Case Study: Exploratory Data Analysis and Natural Language Processing
- Case Study: Google Analytics Audit and Attribution
- Case Study: Natural Language Processing
- Case Study: SEO Audit and Competitive Strategy
Here’s a roundup of who’s hiring, based on positions shared in the Analytics for Marketers Slack group and other communities.
- Amazon Ppc Specialist at Kilo Health
- Analytics Manager at TRKKN
- Campaign Manager at Movable Ink
- Program Manager, Data Engineering at PepsiCo
- Qa Automation Engineer at Kilo Health
- Seo Analyst at Social Driver
- Seo Technical Manager at UMGC Careers
- Sr. Manager, Conversion Rate Optimization at Auctane Careers
Are you a member of our free Slack group, Analytics for Marketers? Join 3000+ like-minded marketers who care about data and measuring their success. Membership is free – join today. Members also receive sneak peeks of upcoming data, credible third-party studies we find and like, and much more. Join today!
Now that you’ve had time to start using Google Analytics 4, chances are you’ve discovered it’s not quite as easy or convenient as the old version. Want to get skilled up on GA4? Need some help with your shiny new system? We can help in two ways:
Where can you find Trust Insights face-to-face?
- ISBM, Chicago, September 2023
- Content Marketing World, DC, September 2023
- Marketing Analytics Data Science, DC, September 2023
- Content Jam, October 2023
- MarketingProfs B2B Forum, Boston, October 2023
- Social Media Marketing World, San Diego, February 2024
First and most obvious – if you want to talk to us about something specific, especially something we can help with, hit up our contact form.
Where do you spend your time online? Chances are, we’re there too, and would enjoy sharing with you. Here’s where we are – see you there?
- Our blog
- In-Ear Insights on Apple Podcasts
- In-Ear Insights on Google Podcasts
- In-Ear Insights on all other podcasting software
Our Featured Partners are companies we work with and promote because we love their stuff. If you’ve ever wondered how we do what we do behind the scenes, chances are we use the tools and skills of one of our partners to do it.
- StackAdapt Display Advertising
- Agorapulse Social Media Publishing
- WP Engine WordPress Hosting
- Talkwalker Media Monitoring
- Marketmuse Professional SEO software
- Gravity Forms WordPress Website Forms
- Otter AI transcription
- Semrush Search Engine Marketing
- Our recommended media production gear on Amazon
Read our disclosures statement for more details, but we’re also compensated by our partners if you buy something through us.
Some events and partners have purchased sponsorships in this newsletter and as a result, Trust Insights receives financial compensation for promoting them. Read our full disclosures statement on our website.
Thanks for subscribing and supporting us. Let us know if you want to see something different or have any feedback for us!
Need help with your marketing AI and analytics?
You might also enjoy:
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, INBOX INSIGHTS. Subscribe now for free; new issues every Wednesday!
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new episodes every Wednesday.