What Is SEO Testing?
SEO testing is the practice of changing one thing on a set of pages, such as a title tag, a heading, a block of content or a group of internal links, and then measuring whether that change moved organic search traffic compared with what would have happened without it. There are two main methods. A split test changes one of two statistically similar groups of pages and leaves the other alone as a control, which large sites can do with real statistical confidence. A before-and-after test changes a page or a small group of pages and compares a period before the change with a period after it, using untouched pages as a rough benchmark, which is what most small business sites can manage. Either way, the point is to replace a general SEO rule with evidence from your own site.

SEO testing is not an SEO audit or a CRO test
The phrase “SEO test” gets used for three different things, and they answer different questions. Free online SEO checkers and audits look at a page and report problems against a list of known issues. A conversion (CRO) A/B test splits visitors on one page between two versions to see which converts better. An SEO test asks whether a change makes Google show and rank your pages differently, which neither of the other two can tell you.
| Method | What it answers | How it works | Who it suits |
|---|---|---|---|
| SEO audit or online checker | “What is wrong with this page?” | Scans the page against a checklist of technical and on-page issues | Any site, as a starting point |
| CRO A/B test | “Which version converts more visitors?” | Splits live visitors on one URL between two versions | Sites with enough visitors to reach a conclusion |
| SEO split test | “Did this change improve search traffic?” | Changes one of two statistically similar groups of pages and compares it with the untouched group | Large sites with many templated pages and steady traffic |
| Before-and-after SEO test | “Did search traffic change after this change?” | Compares a period before with a period after, against pages left alone | Small and medium sites |
The reason SEO testing cannot simply copy a CRO test is that Google indexes and ranks one version of each URL at a time, so you cannot rank half of a page with one title and half with another, which is why an SEO split test divides pages rather than visitors. The detail of that method, including how groups are chosen and compared, is covered in the guide to SEO A/B testing. This page covers the wider idea and the version that works on a smaller site.

Split tests and before-and-after tests
A split test is the stronger method because the control group lives through the same weeks as the changed group. Seasonality, a Google update or a competitor’s campaign affects both halves, so the difference between them is more likely to come from your change. The catch is scale. SearchPilot, a company that sells SEO split-testing software, publishes a rule of thumb that the site section being tested should receive at least 1,000 organic visits a day, not counting the home page, and says that, in general, tests on smaller sections lack the statistical power to detect much (in its guide to minimum requirements, last updated in April 2024). That is a vendor’s guideline rather than a law, but it shows why split testing is mostly a tool for retailers, publishers and directories with thousands of similar pages.
A before-and-after test is what most small and medium business sites can run, since a site that gets nowhere near that much traffic to one section cannot split-test with any confidence. You make a change, wait, and compare. Its weakness is that time itself is a variable: anything else that changed in those weeks shows up in the result. You reduce that risk by comparing the changed pages with similar pages you left alone, by using periods of equal length that cover the same days of the week, and by treating the result as evidence that points one way rather than as proof. A single page with a handful of impressions a month cannot be tested in any meaningful sense, because normal week-to-week movement is larger than any effect you could measure.

What you can test, and what Google decides for you
Almost anything you control on a page can be tested: title tags and meta descriptions, headings, the depth and order of the content, internal links to and from the page, structured data, image alt text, page speed and the layout of a whole template. Title and description tests are the usual starting point because they are easy to change and they target click-through rate, which Search Console reports directly. Content and internal-link tests aim at rankings, which move more slowly and depend on more outside factors.
Two of those starting points come with a caveat Google documents. Google’s page on title links says it uses several sources “to automatically determine the title link”, including the title element, the main visual title on the page and its headings, so the title you test may not be the one searchers see. Its page on snippets says snippets are “primarily created from the page content itself” and that Google sometimes uses the meta description instead when it describes the page better. Before you read anything into a title or description test, search for a few of the pages and check what Google is actually displaying. If it rewrote the title, you tested your title element, not the result people clicked.

How to run an SEO test on a small site
1. Write down the hypothesis. State the change and the result you expect in one sentence, for example “adding the city name to the titles of the six service pages will raise their click-through rate for local searches.” A test without a written prediction tends to be read as a success whatever happens.
2. Choose one metric. Click-through rate suits title and description tests, impressions and position suit content tests, and clicks or conversions suit anything aimed at leads. Picking the metric after the test is over invites you to report whichever one went up. Because CTR also changes when average position or the mix of queries changes, check that position stayed roughly stable before crediting the new wording.
3. Record the baseline. In Google Search Console, filter the Performance report to the pages in the test and export clicks, impressions, CTR and position for a period before the change. Do the same for a set of similar pages you will not touch. Note the limits of the data: Search Console’s average position is the position of the topmost result from your site, and its help pages note that filtering by query or URL “can affect the report totals due to data truncation and the omission of anonymized queries.”
4. Change one thing. Changing the title and the content at the same time leaves you unable to say which one did the work. If you must make several changes, treat the result as a test of the package, not of any single part.
5. Mark the date. Write the date of the change somewhere you will find it, and add a custom annotation to the Search Console chart if your account offers one. You can ask Google to recrawl the changed URLs, but Google’s page on recrawling says crawling “can take anywhere from a few days to a few weeks” and that there is a quota for these requests.
6. Wait for a comparable period. There is no fixed length that suits every test. Google’s own testing page says the time needed depends on factors such as traffic and conversion rates. Start counting only once Google has recrawled the changed pages and, for a title or description test, is showing the new version. Decide the length of the comparison period before you start, keep it the same as the before period, and accept that a page with little traffic may never give a clear answer.
7. Compare and decide. Use the Compare option in the Performance report’s date filter to set the before and after periods side by side, for the changed pages and the untouched ones. If the changed pages improved clearly and consistently while the comparison pages did not, and no update or seasonal swing explains it, roll the change out to a few more pages and keep watching, which is where ongoing SEO monitoring takes over. If nothing changed, or both groups moved together, revert or leave it and record the result so nobody tests the same idea again next year.

What invalidates a result, and Google’s rules for tests
Seasonality is the most common problem in a before-and-after test. As an illustrative example, a Calgary landscaping company that changes its titles in March and compares April with February will almost certainly see more searches, because demand for its services rises in spring, not because the titles improved. Comparing against untouched pages, or against the same months of the previous year, keeps that from being mistaken for a win. Holidays, local events and a run of bad weather can do the same thing over shorter periods.
Google updates are the second problem. Google’s page on core updates advises checking the Search Status Dashboard for an update’s start and end dates and waiting at least a full week after it finishes before analysing your site. If a core update lands in the middle of a test, treat the result as inconclusive unless the changed and untouched pages clearly moved differently, and consider running it again later.
If your test uses separate URLs for each version, or shows different versions to different visitors, Google has published rules in Minimize A/B testing impact in Google Search. Do not cloak: the page says that showing one set of URLs to Googlebot and another to people is against Google’s spam policies whether or not a test is running. Put a rel=”canonical” link on the alternate URLs pointing to the original, which Google prefers to a noindex tag. Use a 302 (temporary) redirect, not a 301, if you send some visitors to a variation URL. Remember that Googlebot “generally doesn’t support cookies”, so it sees the version shown to browsers without them. Finally, end the test and remove its URLs and scripts as soon as you have an answer, because Google says a test that runs for an unnecessarily long time may be treated as an attempt to deceive search engines. A split test that changes real pages, rather than serving variants, does not run into these rules in the same way, since every visitor and Googlebot see the same version of each page.
Tools, and where Google Ads helps
Search Console is free and is enough for a before-and-after test on a small site, with Google Analytics added when the metric is enquiries or sales rather than clicks. A spreadsheet that records each test’s hypothesis, dates, pages, metric and result is worth more than any tool, because it stops the same idea being tested twice. Dedicated platforms such as SearchPilot automate group selection and the statistics for split tests, and they make sense for sites large enough to meet their traffic requirements.
For a business that also runs Google Ads campaigns, there is one more option. Google Ads custom experiments, described in Google Ads Help under Set up a custom experiment, share a campaign’s traffic and budget between the original and a changed version, and for Search campaigns they can split by cookie or by search. That is a genuinely randomized test, and it can show which of two messages searchers respond to, once the campaign has had enough traffic for a clear result. A wording that wins in an ad is a sensible candidate for a title tag test, though not a proven one, since people treat ads and organic results differently and Google may rewrite the title anyway.
When SEO testing is worth it
SEO testing is worth it when a change would affect many pages, when the pages get enough search traffic for a difference to show, and when you would otherwise be acting on a general rule from a blog post. It is not worth it for a single low-traffic page, for fixes that are plainly needed such as a broken page or a missing title, or as a reason to delay work you already know is right. Results are slow either way, because the observation window depends on recrawling, traffic and the size of the effect, and a low-traffic test can stay inconclusive however long you wait.
If you would like a second opinion on what to test first, or on a result you are not sure how to read, SEO Company To-The-TOP! has provided White Hat SEO in Calgary since 2007 and reports client positions monthly. To talk it through, call (403) 308-5949.
Contact Calgary SEO Company To-The-TOP!
Questions about anything in this article, or about your own rankings? Talk to a Calgary SEO specialist directly.
Phone: (403) 308-5949
Address: 1509 14 Ave SW, Calgary, AB T3C 0W4
Hours:
Monday to Friday: 10:00 am – 7:00 pm
Saturday: 12:00 pm – 4:00 pm
Sunday: closed
