SEO Testing
There’s no denying that now when most people want to find something, their reflex is to go looking for it online. Using a telephone to call up information, or even resorting to the old standbys like the thick “Yellow Pages” phone books are things of the past.
Why go through all that when you can pull out your smartphone and type or even verbally ask a question and get an instant answer from a search engine like Google?
For businesses, you’re not the only one searchable on the internet, so are millions of other businesses around the globe.
You’ve gone from having a small, local business that was only accessible to people in a town to be able to service anyone on the planet. So how do you make sure people can find you in a crowded virtual marketplace? With SEO!
If you’re new to search engine optimization or you want to refresh your skills, we’ve put together a guide to help you maximize your SEO potential. But first, let’s go over the basics.
How Search Engines Identify the Best Results
Before you can begin to optimize your website and other content for search engines, you need a basic understanding of search engine functionality.
Google, as the most common example, conducts 40,000 searches per second. It’s not doing that by having a staff of humans conducting manual searches, so how does Google manage?
Search engines use an automated system that relies on an extremely complex set of mathematical formulas known as algorithms.
Algorithms are, at the simplest level, conditions or rules that then build on each other to become more sophisticated and produce the desired results. At the start of the internet age, search algorithms were much simpler, and unfortunately, open to exploitation or abuse.
A “good” web page, back in the day, was defined by algorithms as using a relevant keyword a lot of times.
So, if a financial services website used the word “accounting” five times, and another website used it ten times, the website that used it more was defined as “better” and got a higher position on search results.
Unfortunately, this resulted in some people simply copying a word thousands of times and stuffing it into their website to get higher rankings.
Today’s search algorithms are much more sophisticated and nuanced. Algorithms won’t fall for simple keyword stuffing techniques anymore. In fact, they’re actually quite good at identifying artificial, automated efforts.
Algorithms look at the searcher’s terms and match them with many qualified websites, and then they take other users’ data into account on those sites.
Did previous searchers find that suggested site useful or bounce away quickly? Overall, a search engine pores over a lot of data to submit the optimal results to searchers.
SEO Is Mysterious, But Not Impossible to Understand
The biggest challenge with finding ways to increase your ranking on search engines is that Google isn’t telling anyone exactly what works and what doesn’t.
While Google itself has some suggestions on how to improve search rankings, the key pieces of information, such as what their algorithms are and what they are looking for, are trade secrets that Google won’t share with anyone.
So while they provide good, general advice, such as “focus on quality content,” they do not tell you precisely what variables will raise or lower your ranking.
That is where testing comes in! Not only does Google not reveal the details of their SEO algorithms, but they are also constantly improving and updating the algorithms.
So SEO is a constant “arms race” of discovering new trends that seem to be producing better results and then putting those trends to use for your own content.
So what works now may not work next year, and you’ll need to start tweaking your SEO practices to stay relevant and keep or improve your search rankings.
Collecting Results
For people who have the luxury of dedicated marketing teams and IT staff, you can assign SEO tasks to your professionals and leave them to it.
But if you don’t have the company funds or qualified staff to do this, that doesn’t mean that you’ll never know how effective your SEO efforts are.
It’s quite feasible for any business, medium, small, or otherwise, to conduct a test for the effectiveness of SEO practices. It just comes down to a willingness to learn and an investment in the time required.
If you want to test the effectiveness of your own SEO strategies, you’ll need the right tools and methods or guides to follow for the type of protocol and results that you’re interested in.
Running SEO
1. Form a hypothesis
A hypothesis is a prediction. It’s where you decide what you’re going to change and how you think it’ll affect your website’s SEO visibility.
2. Choose your pages
It’s best practice to only run tests with pages that get a decent amount of organic traffic. This is because you’re not going to learn much from a page that gets little or no organic traffic. It’ll just skew your test results.
You can find pages with decent organic traffic in Google Analytics:
- Go to Behavior > Site Content > Landing pages
- Filter by organic traffic only
- Filter for the types of pages you want to test (e.g., pages with /blog/ in the URL)
- Filter for pages with at least [x] sessions
3. Pull a random sample
You’ll be running an AB split-test, so you’ll only need to make changes to a random subset of pages. The remaining pages will form your control group.
If you’re using a dedicated SEO testing tool, it may handle random sampling for you. However, if you’re doing things manually, the easiest way to pick a random sample is using the “Randomize range” function in Google Sheets.
Here’s how to do it:
- Important your dataset (URLs, etc.) into Google Sheets
- Highlight all the data
- Right-click and hit “Randomize range”
4. Decide on the test duration
Knowing how long to run a test for can be difficult, but it’s basically as long as it takes to gather a decent amount of data. This could be anywhere from a few days to a few months. It depends on how much traffic you get and how many pages you have.
Here are a couple of pointers:
- Run the test long enough for Google to re-crawl the pages in your variant group. You can’t attribute any fluctuations in traffic or rankings to your changes unless the pages have been re-crawled.
- Run the test long enough to get a decent amount of traffic to your pages. Don’t make any conclusions until your pages have received significant traffic, as they’ll likely be statistically insignificant.
5. Decide how to track test results
Before you make changes to your site, you need a way to track results. The easiest way to do this is to use an SEO testing tool. It connects to your Google Search Console account and tracks results for you.
If you want to test changes in organic traffic, use Google Search Console or Google Analytics. Just compare average differences in traffic to both groups for the period before and after the changes were made.
If you want to test changes in CTR, use Google Search Console and compare average changes.
If you want to track changes in organic rankings, use Google Search Console or a rank tracking tool like Ahrefs’ Rank Tracker. The latter is arguably the better option because it gives you actual ranking positions—not just averages.
To set this up in Ahrefs’ Rank Tracker, add and tag two batches of keywords:
- Control group keywords tagged as “control”
- Variant group keywords tagged them as “variant”
You can then see average ranking positions for the keywords in each group in the Tags report:
6. Make the changes
It’s finally time to implement your AB test. Here are a few pointers for doing this:
- Don’t make changes to pages in the control group. It’s called the control group for a reason; it’s your baseline against which you’ll evaluate the impact of your change.
- Change one variable only. If you change multiple things, you’ll have trouble figuring out whether it was one or a combination of things that led to the outcome.
- Keep a record of the changes you made and the date you made them.
- Make sure you have a way to revert back. You’ll need to do this if your SEO test produces an inconclusive or negative outcome.
- Don’t test pointless things. If you can quickly and easily implement an SEO best practice, don’t bother testing it—just do it.
7. Check the results
If you’re using a testing tool, it’ll probably produce some nice graphs for you to analyze once the test is over.
If not, you can evaluate the outcome of your test manually using Google Analytics (if you’re testing for changes in traffic), Google Search Console (if you’re testing for changes in traffic, rankings, or CTR), or a rank tracking tool (if you’re testing for changes in rankings).
If you’re testing for changes in rankings and everything’s set up in Ahrefs’ Rank Tracker, as shown in step five, you can see changes in tracked keyword visibility since the start of your test in the Tags report. Just choose the timeframe.
SEO test samples
1. Testing the impact of title tag changes on CTR for e-commerce product or category pages
As title tags show up in the search results, they can affect your pages’ clickthrough rates (CTR). If you use the same formula for all product or category pages like most ecommerce sites, testing the impact of a different formula on CTR is pretty straightforward.
How to test
Take a bunch of e-commerce category pages that get a decent amount of organic traffic, take a random sample, then change the title tag formula on those pages.
How to measure
Use Google Search Console to compare the average CTR for pages in both groups for the period before and after the changes. You’ll probably need to export this data to a spreadsheet or Google Data Studio as the filters in Search Console leave a lot to be desired.
2. Testing the impact of a content optimization tool on blog post rankings
Many SEO professionals swear by content optimization tools—but many others don’t. If you’re in the camp that thinks there might be something to this approach, why not test it?
How to test
Take a bunch of blog posts that get a decent amount of organic traffic and rank in the top 10 for their main target keywords, take a random sample, then “optimize” those posts with a content optimization tool.
Here’s how to find such blog posts in Ahrefs’ Rank Tracker:
- Choose your project
- Go to the Pages report
- Filter for top 10 rankings
- Filter for pages with at least [x] monthly estimated organic traffic from tracked keywords
- Export the results
How to measure
Use Ahrefs’ Rank Tracker to compare the average visibility for tracked keywords in each group on the test start and end dates as explained in step five.
3. Testing the impact of adding snippet-friendly content on blog post featured snippet rankings
Ranking in featured snippets usually drives more clicks. So if you have blog posts targeting and ranking for keywords that trigger featured snippets, why not test your ability to win them?
How to test
Take a bunch of blog posts that get a decent amount of organic traffic and rank in the top 10 for keywords that trigger featured snippets, take a random sample, then add snippet-friendly content to those posts.
Here’s how to find such posts in Ahrefs’ Site Explorer:
- Enter your blog/website into Site Explorer
- Go to the Organic Keywords 2.0 report
- Filter for top 10 rankings
- Filter for keywords that send at least [x] monthly estimated organic traffic (so that you know the pages get a decent amount of traffic)
- Filter for unowned featured snippets
- Export the results
How to measure
Use Ahrefs’ Rank Tracker to compare the number of owned featured snippets for tracked keywords in each group on the test start and end dates.
Conducting a Split Test for SEO
In an ideal, lab situation, you would have two identical samples. You would conduct your test on one while leaving the “control sample” alone, to use as a basis for comparison and see what differences occur. But that’s not practical in the real world with a live website.
If you have similar web pages, such as a cooking website with many recipes, you can see what happens if you leave one web page alone, with your standard template, and tweak another web page with the updates you want to test. Then, over time, compare the data.
Though that method works, the ultimate test would be on the same webpage. So look at the results for your cookie recipe webpage before you begin. Then enact your changes, and use your tools to measure rates of visitation, click-throughs, and other factors.
Alternate between the original and tweaks for a week or two at a time to carefully observe your metrics. Doing this in different configurations should eventually give you a much clearer idea of how your SEO tactics are working, or not working.
Periodically testing your SEO effectiveness is something every website owner should be doing on a regular basis. That is especially true once you have made changes or are adding new content. Every change you make can affect your rankings.
Side notes
If you’re running tests and making conclusions after only a few hundred visits, chances are the results aren’t going to be statistically significant.
