Optimizing Ideation: Using Data to Drive Better Brainstorming

by Dec 21, 2017

When I first started A/B testing 15 years ago, our focus was purely conver­sion rate opti­miza­tion with no concern about scal­able learn­ing and little to no under­stand­ing of how to use data to provide support­ing evidence for test ideation and prior­i­ti­za­tion. It was very common for me to sched­ule a brain­storm­ing session with a few key stake­hold­ers where we would liter­ally sit in a room and throw out ideas we could test. We would debate the ideas with a focus on feasi­bil­ity and I would leave the meeting with a list of tests to go run. Unfor­tu­nately, we didn’t get great or even ok results most of the time. Of course, there would always be that one big win that would keep us coming back for more, but we never seemed to have wins large enough to outweigh the nega­tive impact of all those losses.

It took a few years and a job at a new data-driven company before I started apply­ing data to my brain­storm­ing process. Even so, it was still the same basic prin­ci­ple: get every­one in a room and throw out test ideas. The only differ­ence was that, this time, we used data to iden­tify prob­lems and the ideas we brain­stormed were about solu­tions. Now we were using data to make sure we were solving real prob­lems. Unsur­pris­ingly, we saw much better results and could drive real improve­ment. And because we were using data to iden­tify customer pain points, we were also address­ing prob­lems our customers were having. Much better!

But some­thing was still missing. Using analy­sis to iden­tify prob­lems is great, but it will never make you better. It will only help you fix what’s broken. What happens when you’ve run out of things that need fixing? I know, I know … never should happen, but at some point, the juice just might not be worth the squeeze. And if you use data to iden­tify some­thing broken, why test it when you could just fix it? I was think­ing these Jack Handey-esque deep thoughts about five years ago when it hit me: what if we flipped things around and used brain­storm­ing to iden­tify prob­lems and data to suggest solu­tions?

In this new world scenario I had created, it would be easy for us to utilize exist­ing use cases or create new ones, and then run through the site expe­ri­ence from the perspec­tive of those who would fit in those use cases. We could then iden­tify areas of oppor­tu­nity to make the site expe­ri­ence better for the customer (and more lucra­tive for the busi­ness). Once we iden­ti­fied areas to eval­u­ate, we could dive through the avail­able data (CRM, web behav­ioral, VoC, etc.) to develop poten­tial solu­tions to test. We were shift­ing our focus from fixing things that were broken to truly opti­miz­ing the customer rela­tion­ship with the brand.

It took just a few trials through this new process to under­stand its poten­tial and realize its benefit. But we’re in the busi­ness of contin­u­ous opti­miza­tion, right? So we can’t stop there! On a whim, I decided to see what would happen if I combined elements of all of these method­olo­gies and I struck gold.

Let me break down what this looks like in prac­tice. On a regular cadence, we meet for opti­miza­tion brain­storm­ing. Analysts are asked to prepare data iden­ti­fy­ing poten­tial fric­tion points, busi­ness owners are asked to do a site run through using various use cases to iden­tify oppor­tu­ni­ties or issues. All come together to present their iden­ti­fied prob­lems and/or proposed solu­tions. Analysts take all ideas and/or prob­lems currently without data support as take­aways to conduct the requi­site analy­sis to back up the solu­tion and/or propose ideas for iden­ti­fied oppor­tu­ni­ties or prob­lems.

At the next regu­larly sched­uled brain­storm­ing session, the analysts come with what they learned about the iden­ti­fied ideas brought up in the prior meeting and busi­ness owners come with new areas to assess … and a cycle is born! Now, we get all the best ideas from all possi­ble areas and we use data both to iden­tify prob­lems and oppor­tu­ni­ties! Plus, there’s the added benefit of teach­ing the entire orga­ni­za­tion to be data-driven in their deci­sion-making. And the outcome? The most success­ful opti­miza­tion program I’ve ever been a part of to this day.

Want help with your opti­miza­tion ideation process?
Search Discovery can help develop a new process or improve an exist­ing process from idea intake to prior­i­ti­za­tion. Hit us up!