Comparative Analysis: Deliver Your Strongest Tests

Use comparative analysis to research a range of sites, understand familiar experiences customers are likely to expect, and to power & mature your optimization program.
This is the third post in a series that outlines ideation techniques to develop customer-centric test hypotheses. Use these methods to power your experimentation platform and your a/b testing program.

This blog series complements our free, in-depth guide to using structured ideation methods to generate better ideas, better tests, and better decisions.

All You Need to Know about Powering Your Tests with High-Value Ideas.


Each post in this series highlights another chapter of the e-book. We kicked things off in the first post by setting up the concept of structured ideation and the importance of using quality research to generate evidence-based ideas and recommendations.

The second post focused on digital behavioral analysis, one of six structured ideation techniques examined in the guide. Digital behavioral analysis uncovers patterns in how users interact with your website and pinpoints potential areas of improvement.
Below, we add another approach to your structured ideation toolkit—comparative analysis.

Below, we add another approach to your structured ideation toolkit—comparative analysis.

Chapter 3: Comparative Analysis

Comparative analysis (or, interchangeably, “comparative research or competitive analysis”) shifts the focus from how users interact with your site to their experiences on a range of other sites. This ideation technique gathers valuable information about content, design, industry tactics, and overall experiences that help shape customer expectations.
The third chapter of the guide takes an in-depth look at the concept of comparative analysis, the steps involved in performing this research, and the pros and cons of this method. We provide concrete examples and downloadable templates to lead you through your own comparative research.

Comparative analysis comprises four components: planning, setup, analysis, and ideation proposals. The first step, planning, ensures your comparative analysis generates the most useful results. You begin by identifying the problem to solve and establishing the business objective for the information you want to gather. For example, you may want to address an issue you uncovered during your digital behavioral analysis and focus your comparative analysis on techniques other companies are implementing to generate results.

In the setup phase, you create a list of company websites to evaluate. These include direct competitors, as well as players outside your industry that are deploying innovative strategies. You’ll also identify the specific themes or practices that you want to investigate on each site.
In the analysis phase, you review the tactics used by other companies and score them based on usability and innovation. Our white paper includes a downloadable comparative analysis spreadsheet that helps you gather insights and formulate findings across a diversified yet relative list of companies that you’re evaluating.

During the ideation proposal phase, you build hypotheses based on your research. A well-executed comparative analysis can deliver up to 15 solid hypotheses backed by qualitative research. We recommend structuring these using the IF THEN BECAUSE format, summarizing what you want to test, what you believe it will impact (primary KPI), and why you believe this tactic will work (logic or reasoning behind your hypothesis).

Maximizing Optimization Program ROI with Search Discovery & Conductrics

A robust optimization program moves away from occasional a/b testing, instead using a deliberate, scientific process to enrich customer experiences, boost conversions, and improve cost-effectiveness. Comparative analysis, and the other structured ideation methods in the guide, should be a part of your program’s ongoing idea generation processes.

Get the most ROI from your experiments by taking your optimization program to the next level. Our powerful partnership with Conductrics leverages expertise and innovative technology to expand your experimentation program’s impact.

Search Discovery’s research and experimentation experts help design processes and automations that guide your program, ensuring experiments are structurally sound and impactful for your business goals. Conductrics provides the a/b testing, machine learning, and optimization technology that powers these processes, with full-stack integrations on dedicated servers and the flexibility to meet your unique needs.

Contact us today to learn more about how comparative analysis can help you create a robust optimization program.

Leave a Comment

Your email address will not be published.

Contact Us

Related Posts

Join the Conversation

Check out Kelly Wortham’s Optimization based YouTube channel: Test & Learn Community.

Search Discovery
Education Community

Join Search Discovery’s new education community and keep up with the latest tools, technologies, and trends in analytics.




Scroll to Top


Catch the latest industry trends we’re watching and get new insights from our thought leaders delivered directly to your inbox each month.