INBOX INSIGHTS: Disadvantages of Predictive Analytics, SEO A/B Testing (1/5) :: View in browser
Disadvantages of Predictive Analytics
In 2018, back when I was fresh-faced, full of energy, and had an endless well of optimism, I wrote this blog about the issues with predictive analytics.
Now, 3+ years later, when I have permanent circles under my eyes, I never know what day it is, and an ounce of optimism sounds sarcastic, I want to revisit this post to see what’s changed.
In 2018, I talked about Black Swan Events. Oh, how little 2018 Katie knew. She did not know that in 18 short months she would be in a global pandemic. Therein lies the definition of a Black Swan Event – it’s something unforeseen.
The disadvantage of predictive analytics almost two years into a global pandemic is that most historical data that pre-dates March 2020 is irrelevant. So much has changed about how we live our lives, how supply and demand operate, and how we view the world, that using that data won’t give you a sense of how to plan your marketing.
How do we overcome this disadvantage that is throwing off our analysis?
Look at more than one data source
Sure, you can look at only your data. However, it might not tell you the full picture of what is going on. You may also consider looking at other publicly available data sources as a gut check. Looking at outside data sources can help color the story of what’s going on within your business. Try thinking beyond the digital channels. Here are some examples:
Data.gov is the US Governments open data. You can do research on topics such as local government and climate issues.
Bureau of Labor Statistics can keep you updated on the job market. If unemployment rates go up, there are certain products and services that will cease to be viewed as necessary.
Federal Reserve Economic Data (FRED) can help you understand economic factors that can help predict consumer behavior.
Forecast shorter periods of time
Given that just about every dataset either spiked or declined in March of 2020, consider shorter time periods for your predictions so that you can remain agile. I would not recommend using a predictive forecast for an entire year’s worth of planning. You can use it as a guide, but know that the data will likely change. A better plan is to look at quarterly, monthly, or even weekly trends if you have the resources to do so. Doing this will allow you to make changes faster as you learn what’s working.
Some trends don’t change
Despite the pandemic, humans are creatures of habit. We still make New Year’s resolutions, we still celebrate holidays, we still take summer vacations. What’s changed is how we approach these things. As you’re looking at your keywords and forecasting, be sure to include qualifiers such as “virtual”, “remote” and “alternatives”.
Are you looking for some help with your planning and want to see how a predictive forecast fits into your marketing? Give us a shout!
– Katie Robbert, CEO
Do you have a colleague or friend who needs this newsletter? Send them this link to help them get their own copy:
In this week’s In-Ear Insights, Katie and Chris discuss the basics of local SEO. Learn what you need to know about the 4 key areas of local SEO. Watch/listen to this episode of In-Ear Insights here »
This Thursday at 1 PM Eastern, we’ll be looking at email marketing best practices for the new year including list hygiene and re-engagement.
Need a reminder?
Here’s some of our content from recent days that you might have missed. If you read something and enjoy it, please share it with a friend or colleague!
- 12 Days of Data 2021, Day 1: Instagram for Brands
- 12 Days of Data 2021, Day 2: Instagram for Influencers
- 12 Days of Data 2021, Day 3: Instagram Media Types
- 12 Days of Data 2021, Day 4: Tiktok Trending Videos
- 12 Days of Data 2021, Day 5: TikTok Marketing Topic Deep Dive
- 12 Days of Data 2021, Day 6: SEO Page Decay Statistics
- 12 Days of Data 2021, Day 7: Press Release Statistics
- 12 Days of Data 2021, Day 8: Press Release (Over)used Words
- 12 Days of Data 2021, Day 9: 2022 Marketing Campaign Forecast
- 12 Days of Data 2021, Day 10: Top News and Web Content
- 12 Days of Data 2021, Day 11: Content Republishing
- 12 Days of Data 2021, Day 12: Slack and Discord
Get skilled up with an assortment of our free, on-demand classes.
- What, Why, How: Foundations of B2B Marketing Analytics (new!)
- How to Think About Google Analytics 4 (new!)
- Fundamentals of Marketing Analytics (new!)
- How to Deliver Reports and Prove the ROI of your Agency
- Powering Up Your LinkedIn Profile (For Job Hunters)
- Competitive Social Media Analytics Strategy
In this week’s Data Diaries, we want to know the impact of an SEO change we made. Back in November, we switched out the Yoast SEO plugin with a different plugin, the RankMath plugin. The challenge with this sort of marketing technology change is that you can’t do any kind of A/B testing – you can’t show a different website to Google and other search engines part of the time and show a differently-optimized version other times.
So, how do we evaluate whether a piece of software is working or not when we can’t do a controlled test? We have to rely on statistical methods that simulate a retroactive A/B test. Here’s how they work, roughly.
We have a series of days called treatment days – days when our change is in effect, like every day since November 29, 2021 when we installed and deployed the new software.
We also have a series of days called control days – days when the change was not in effect, which was every day before November 29, 2021.
We gather up our data for all those days; in this case, we’ll use data from Google Search Console.
When we use statistical software like the R programming language to perform a technique called propensity score matching, what the software does is look at our treatment days and try to match them as closely as possible with similar control days. For example, we’d want the software to compare similar days of the week; there’s no sense in comparing a Sunday to a Tuesday.
This technique also allows us to account for other things going on, which is the most critical part. For example, if we were running Twitter ads during our treatment period, we’d want to compare the treatment period to control period days when we were also running Twitter ads.
With all that in mind, what does it look like when we examine our analysis of our Google Search Console data with the new WordPress plugin versus the old one?
What we see here are a series of outcomes; the second column shows us the mean (average) for days of our treatment, while the third column shows us our control days. The fourth column shows us the differences. Let’s run through them:
- For days using the new plugin, we see a 16% increase in the number of search impressions. This is really important – impressions means Google sees our site as relevant to what searchers were looking for and displayed us in the search results.
- We also see a 10% increase in overall clicks on our search results. Again, good news.
- We see no difference in the distance metric – this is a statistical measure to show us how different our control days are from our treatment days. If it were a really big number, the test might not be statistically valid.
- We see no difference in clickthrough rates; this makes logical sense because even though our clicks increased, so did our impressions.
- We see no difference in our ranking positions. This is more of a function of having relevant content than it is search optimization, something that a change in marketing technology would not impact substantially.
- Finally, we see no difference in the numerical assignment for day of week, nor should we.
This analysis tells us that there is a meaningful difference for using the new plugin over the old one. With more than a month of data under our belts, I feel confident that the change is worth keeping.
What’s the key takeaway, the So What? moment? It’s not which plugin to use – it’s that it is possible in some circumstances to conduct an A/B test retroactively if you have the right data and the right tools. Study this method and apply it to your marketing (or ask us to do it for you, I guess).
- Case Study: Google Analytics Audit and Attribution
- Case Study: Natural Language Processing
- Case Study: SEO Audit and Competitive Strategy
This is a roundup of the best content you and others have written and shared in the last week.
SEO, Google, and Paid Media
- How To Find SEO and Keyword Ranking Success on Google in 2022
- 5 On-Page SEO Factors To Check In Underperforming Content
- Wrapping up 2021 with our top 10! via Search Engine Watch
Social Media Marketing
- How to Make and Sell Merch on TikTok (2022) via Shopify Canada
- 30 Social Media Video Statistics to Guide Your Strategy via Sprout Social
- Marketers Need to Think Granular and Niche to Grow on TikTok
- Is Your Content Marketing Worth Stealing?
- How to Measure the Effectiveness of Your Training and Support Content via TechSmith Tutorials
- 10 global PR trends to look out for in 2022 via Agility PR Solutions
Data Science and AI
- The Complete Guide to AI for Businesses and How Its Making a Difference
- Preparation is key to AI success in 2022 via VentureBeat
- A First-Principles Theory of NeuralNetwork Generalization The Berkeley Artificial Intelligence Research Blog
Are you a member of our free Slack group, Analytics for Marketers? Join 1900+ like-minded marketers who care about data and measuring their success. Membership is free – join today. Members also receive sneak peeks of upcoming data, credible third-party studies we find and like, and much more. Join today!
- Are you happy with the answers you’re getting out of Google Analytics today?
- Are you confident in the decisions you’re making from Google Analytics?
- Are you secure in the knowledge that you’re set up properly to benefit from Google Analytics 4?
If you answered no to any of the questions above, let’s take some time to get your analytics in shape. Google Analytics 4 is the analytics software of the future. Google has made no bones about the fact that anything new they develop will be in GA4 only. To take advantage of the new features, you need to get your house in order now.
So we’re offering you a Google Analytics 4 bootcamp. We’ll help you:
- Get your existing Google Analytics account in shape with proper goals, tracking cleanup, and best practices
- Identify key issues that will block your ability to use Google Analytics 4 and help resolve them
- Help set an analytics strategy for this that focuses on answers, decisions, and growth rather than pouring more data in your inbox
- Build a migration plan for Google Analytics 4, including Google Tag Manager and Google Data Studio
Interested in sponsoring INBOX INSIGHTS? Contact us for sponsorship options to reach over 22,000 analytically-minded marketers and business professionals every week.
Where can you find us in person?
- MarketingProfs B2B Forum, April 2022, virtual
- MarketingProfs B2B Forum, October 2022, Boston
First and most obvious – if you want to talk to us about something specific, especially something we can help with, hit up our contact form.
Where do you spend your time online? Chances are, we’re there too, and would enjoy sharing with you. Here’s where we are – see you there?
- Our blog
- In-Ear Insights on Apple Podcasts
- In-Ear Insights on Google Podcasts
- In-Ear Insights on all other podcasting software
Our Featured Partners are companies we work with and promote because we love their stuff. If you’ve ever wondered how we do what we do behind the scenes, chances are we use the tools and skills of one of our partners to do it.
- Hubspot CRM
- StackAdapt Display Advertising
- Agorapulse Social Media Publishing
- WP Engine WordPress Hosting
- Techsmith Camtasia and Snagit Media Editing
- Talkwalker Media Monitoring
- Marketmuse Professional SEO software
- Gravity Forms WordPress Website Forms
- Otter AI transcription
- Semrush Search Engine Marketing
- Our recommended media production gear on Amazon
Read our disclosures statement for more details, but we’re also compensated by our partners if you buy something through us.
Some events and partners have purchased sponsorships in this newsletter and as a result, Trust Insights receives financial compensation for promoting them. Read our full disclosures statement on our website.
Thanks for subscribing and supporting us. Let us know if you want to see something different or have any feedback for us!
Need help with your marketing data and analytics?
You might also enjoy:
Get unique data, analysis, and perspectives on analytics, insights, machine learning, marketing, and AI in the weekly Trust Insights newsletter, Data in the Headlights. Subscribe now for free; new issues every Wednesday!
Want to learn more about data, analytics, and insights? Subscribe to In-Ear Insights, the Trust Insights podcast, with new 10-minute or less episodes every week.