For years, SEOs (unlike our paid media counterparts) have dealt with less and less data.
First, Google stopped providing keyword data from organic search.
Then, Google made Keyword Planner a lot less useful with averaged and rounded volume and merged queries.
Then, Google made Keyword Planner a tool specifically for active advertisers.
In fact, there was a period in 2015 when keyword research moved from an art + science to a total art.
And then came clickstream data. It was like magic.
In fact, the past 4 years have arguably been as rich in keyword data (especially when you add in the improved Search Console) as pre-2013 when you had automated rank checkers and keyword data directly in Google Analytics.
But like a lot of SEOs, I didn’t dig too far into the details. The data was accurate enough. It led to “paint by the numbers” keyword research* via tools like Ahrefs and SEM Rush.
*Example – search keyword in Ahrefs, sort by phrase match, filter by Keyword Difficulty less than 15, then pick the highest volume.
But details matter, especially when it comes to such critical marketing data. Here’s what Clickstream data is and is not based on my research.
What is Clickstream Data?
Clickstream Data is the sequence of hyperlinks one or more website visitors follow on a given site, presented in the order viewed – according to Wikipedia.
In other words, Clickstream data is not one thing really, it’s more of a category of data. Clickstream data is any data that captures a single user’s journey around a website or the Internet.
If you have ever seen a website heatmap or Google Analytics flowchart or a website usability test, then you have seen Clickstream data in action.
Clickstream data is literally just the stream of clicks that a user makes as they journey around the Internet.
How Is Clickstream Data Used for SEO?
Clickstream data has a lot of indirect uses for SEO. For example, the Clickstream data from Google Analytics or Crazy Egg can help you improve your content and landing pages.
But the real goldmine is Clickstream data from Google Search Engine Result Pages (SERPs).
If you can see what a user searches for, what results show up, and what the user clicks on – you can gain a lot of insight.
In fact, that data is so insightful that the US Department of Justice used it within its anti-trust probe of Google.
Without Clickstream data, most SEOs can only extrapolate ballpark traffic numbers from Google SERPs.
Without Clickstream data, most SEOs judged the value of a #1 ranking or rich snippet of all possible queries based on a few narrow, biased usability studies.
With a large enough Clickstream data set, you could actually say what queries drove traffic, what queries drove revenue, and what queries had nuance. You could see everything. Like I outlined in my Ahrefs Review, the only real limit to keyword research is your imagination & creativity.
The data is cleaner, better, and more sustainable than data pulled from web scrapers breaking Google’s terms of service. And it’s much more accurate and specific than data pulled from Google Ads API.
Where Does Clickstream Data for SEO Come From?
“But hold up…,” you might be thinking, “…how does one obtain real-user Clickstream data on Google’s SERPs?
Good question!
Actually, it’s a question that is best left unasked. You probably know the answer, but since it’s a bit uncomfortable, most SEOs don’t bother to ask.
Google SERP Clickstream data comes from browser software spying on everyday users while they go about their daily life on the Internet.
Now, before we move to the next section, note that nothing illegal is going on. Everything is above board in the SEO world…it’s just another part of our strange new world of surveillance capitalism.
It’s like location data or Facebook’s social graph on your plate. It’s amazing and legal…just don’t think too hard about it.
Who Provides Clickstream Data for SEO?
There are a few players out there. Most are super-secretive, because, well, the biggest one ceased operations in January 2020 due to a major publicity scandal.
JumpShot was the biggest, best, and most comprehensive Clickstream provider on the Internet. They were wholly owned by Avast. Avast is one of the largest anti-virus and security software companies in the world.
Back in 2015, they started scanning URLs and webpages for security threats. Since they were scanning these URLs anyway, they had the idea to get people to consent to a data sharing agreement.
The data sharing agreement said that Avast would take their browser information, anonymize it, aggregate it, and sell it on to whoever wanted to buy it.
Avast eventually had a global Clickstream data source of 100 million+ users.
Their customers had cheap or free security software.
And companies ranging from Home Depot to Ahrefs to Moz to every other inbound marketing software provider had access to crazy detailed Clickstream data from Google SERPs, YouTube, Amazon, and anywhere else of interest on the Internet.
It’s a 21st Century Surveillance Capitalism Win/Win/Win arrangement.
In many ways, it’s pretty brilliant. Businesses & customers are at the mercy of the ultimate surveillance company (Google) who has all the data.
By combining and anonymizing data from willing users, businesses could get Clickstream data to build better websites while supporting a whole ecosystem of independent marketing businesses. It’s the argument that Rand Fishkin forcefully made after the event.
What Happens if Clickstream Data Disappears?
Clickstream data has basically disappeared as of January 2020.
Jumpshot, the largest provider of Clickstream data, ceased operations immediately on January 30 2020 after an expose from Vice & PC Magazine went viral and started seriously affecting their stock & security subscriptions.
Now, there are still other small providers out there. There are also ad hoc agreements among large agencies and software providers to share the data that they each have. And there are companies that aggregate some smaller, less statistically significant panels of data (e.g., Alexa and SimilarWeb).
But there’s nothing like Jumpshot available. Most keyword research tools are now completely reliant on Google Ads API and manual web scraping.
What Are Some Alternatives to Clickstream for SEO?
To begin, note that your SEO tools are not worthless. Ahrefs still has more link data than anybody else. You can still do amazing things with the tool. KeywordTool.io still has a good web scraper. AnswerThePublic has excellent keyword analysis. Moz still has powerful local tools. SEM Rush still has all their Google Ads data.
The part that has changed is search volume numbers, search snippet performance, SERP click data, etc.
In other words, keyword research is once again a pure art rather than rigorous data science.
Building an alternative to Jumpshot is above my pay grade. I’m sure some data company could do it. I’m thinking Facebook, Equifax, Axciom, AdBlock Plus, or some other software company with enough political and/or public relations expertise could pull it off.
But until then, here are a few ideas to replace Clickstream data & build resilience around your keyword research.
Diversify Your Premium Toolset
First, if you have budget, diversify your SEO tools. If you can afford both Ahrefs and SEM Rush, then having both data sets will put you ahead of the competition.
Learn Different Methods
Second, learn to use non-traditional keyword research methods. I’ve written about mining Google Books, Display Planner, Pinterest, Reddit, YouTube, even Slideshare, and Wikipedia. Use them.
Find Specialized Tools for Specific Needs
Third, learn to use lots of different keyword tools in the right sequence. Keyword tools are a dime a dozen. Check them all out and use them.
Go Old School & Look at the SERPs
Fourth, instead of looking at click through rate and keyword difficulty, go back to looking at the actual SERPs.
Ask yourself if you could (or should) deserve to rank. Install the Ahrefs browser extension and look at the link profile of the ranking URLs.
Learn to Love Search Console
Fifth, start using your Search Console data. It’s really good. And it comes directly from Google. Download 16 months worth and mine it for ideas.
Learn to Make Use of Keyword Planner
Sixth, go back to using Keyword Planner. Get an active Google Ads account. Sure, it’s imprecise. But Google wants everyone to address topics anyway. Use it to find topics, then manually drill down to your keyword theme.
Build a Brand w/ Actual Content
Seventh, do what you were supposed to be doing anyway – building a brand and building links. I know that I have gotten stuck in a keyword-driven rut.
Maybe the absence of paint by the number keyword research tools will drive the DemandMedia 2.0 websites out of business and allow thoughtful, thorough content to rise to the top.
Next Steps
Clickstream data is an incredible tool. It has made SEOs, in particular, more productive and better informed for years.
But like any good tool, it can become a crutch that prevents learning new & better ideas. The interruption of Clickstream data might change the keyword research process of SEOs, but it still won’t stop the need.
