Welcome to our FREE optimization tool hub! We designed these tools to help you make better testing decisions and educate your team and stakeholders. Be sure to check out accompanying blog posts and video tutorials.
Since we receive regular requests for updates, we update these tools regularly! We also create new tools as we uncover needs, so bookmark this page and check back often or fill out the contact form below to be notified of updates and additions,
This simulator is designed to illustrate the difference between an observed result in an A/B test and the actual/true value (which is completely unknowable, but extremely useful to think about).
Experimentation provides you with data by which to make decisions when the truth is unknowable. This simulator is a playground for experimenters to gauge the value of what they’re doing.
Conduct your own statistical analysis and data visualization for a robust view of test performance. You can brand-customize outputs, including experiment visuals, to make handy learning-library artifacts.
Provides experimenters with a responsible way to “peek” at test results and optimize test run times. Supports both test planning and test analysis in one interface. Learn more in this blog, and check out the video tutorial!
Extremely flexible calculator that lets you pull various levers to determine sample size and account for different types of tests, different levels of risk tolerance, and different business needs. Check out this blog to learn more.
Helps you determine what lift you would need to achieve to attain statistical significance within a set timeframe. This calculator is often used in conjunction with the Sample Size or Sequential Calculators. Read more about it here.
Bayesian testing usually involves a great deal more complexity, and a huge range of approaches when interpreting the outcomes. Embedded at each step are significant (and sometimes arbitrary) methodology choices that impact outcomes. While we recommend working with a statistician or data scientist when designing and analyzing a Bayesian test, this calculator can help those who want to play around and learn more about Bayesian testing.
This simulator is designed as an educational tool to illustrate the magic of randomization in A/B testing and the impact of blocking on your sampling distribution and ability to control for Type II errors (ie. failing to detect a treatment effect when one is present).
SAVE TIME. WORK SMARTER.
We can transform your business with an innovative technology or help you get the most from what you have. Let’s talk.
