OPTIMIZATION:
Program Health Measurement
WE DELIVER:
Resources and tools to enable a capacity building approach and operational improvements
Identification of barriers to fully realizing success through cross-functional workshops
Defined workstreams and activities that can help overcome the barriers identified
Identification of opportunities to improve the customer/client experience
Focus on initiatives/processes that result in cost savings, efficiencies, and corporate growth
SUCCESS STORIES
OUR APPROACH
We challenge our clients to set goals, measure performance regularly, and adjust and iterate with the objectives of improving your optimization program’s process efficiency and supporting continuous program development. To measure progress and track performance, we enable you to get a holistic view of your optimization programs performance to:
- Minimize the burden of program tracking by implementing automated testing processes
- Identify phases causing experiment bottlenecks to increase efficiency
- Document inputs and learnings of test and identify next action
- Provide a testing roadmap to accurately and holistically display test funnel/pipeline
- Provide high-level scorecard metrics to track progress and enable reporting
LATEST INSIGHTS:
Sample Ratio Mismatch: What is it? Do I have it? How do I fix it?
Learn from Lukas Vermeer’s conversation with the Test and Learn Community about Sample Ratio Mismatch (SRM) or what he calls “one neat trick to run better experiments.” Here’s a simple tool to test for data quality issues. The Test and Learn Community met with the Director of Experimentation at Booking.com, Lukas Vermeer, on a beautiful
How to A/B Test Statistical Intuition via Simulation. Bonus! Free A/B Test Simulation Tool Included!
Developing a solid intuition about the statistical concepts that drive the interpretation of a basic A/B test is a powerful thing. Explore an A/B test simulator to lock in that intuition!
A Cranium-Expanding Machine Learning Discussion with Matt Gershoff
We recap Matt Gershoff’s machine learning discussion with the Test and Learn Community. Read more to see if you’re ready to invest in machine learning!
How (and Why!) To Monitor Your Tests
Common Mistakes When Running A/B Tests There are a few. When running a test, there are multiple places where one can misstep to put the test, and the program, in jeopardy: Peeking at your test. If you have set up a beautiful fixed horizon A/B test, but then you peek for results and call it
