Optimization Archives | Search Discovery

For advertisers, the new loss of third-party cookies creates gaps in media attribution models, so media result measurements become inaccurate. A randomized Controlled Trial (RCT, or “Randomized Control Testing”) is a scientific method we use to solve these problems for marketing teams.

Question: Four test subjects walk into a bar. The 1st wears a shirt and shoes. The 2nd wears a shirt but no shoes. The 3rd wears shoes but no shirt. The 4th wears no shirt and no shoes. Which ones got a drink?
Answer: Unfortunately, none of them accepted cookies, so we’ll never know.

Learn from Lukas Vermeer’s conversation with the Test and Learn Community about Sample Ratio Mismatch (SRM) or what he calls  “one neat trick to run better experiments.” Here’s a simple tool to test for data quality issues. The Test and Learn Community met with the Director of Experimentation at Booking.com, Lukas Vermeer, on a beautiful …

Sample Ratio Mismatch: What is it? Do I have it? How do I fix it? Read More »

Developing a solid intuition about the statistical concepts that drive the interpretation of a basic A/B test is a powerful thing. Explore an A/B test simulator to lock in that intuition!

We recap Matt Gershoff’s machine learning discussion with the Test and Learn Community. Read more to see if you’re ready to invest in machine learning!

Common Mistakes When Running A/B Tests There are a few. When running a test, there are multiple places where one can misstep to put the test, and the program, in jeopardy: Peeking at your test. If you have set up a beautiful fixed horizon A/B test, but then you peek for results and call it …

How (and Why!) To Monitor Your Tests Read More »