Optimizing Ideation: Using Data to Drive Better Brainstorming

by | Dec 21, 2017

When I first start­ed A/B test­ing 15 years ago, our focus was pure­ly con­ver­sion rate opti­miza­tion with no con­cern about scal­able learn­ing and lit­tle to no under­stand­ing of how to use data to pro­vide sup­port­ing evi­dence for test ideation and pri­or­i­ti­za­tion. It was very com­mon for me to sched­ule a brain­storm­ing ses­sion with a few key stake­hold­ers where we would lit­er­al­ly sit in a room and throw out ideas we could test. We would debate the ideas with a focus on fea­si­bil­i­ty and I would leave the meet­ing with a list of tests to go run. Unfor­tu­nate­ly, we didn’t get great or even ok results most of the time. Of course, there would always be that one big win that would keep us com­ing back for more, but we nev­er seemed to have wins large enough to out­weigh the neg­a­tive impact of all those loss­es.

It took a few years and a job at a new data-dri­ven com­pa­ny before I start­ed apply­ing data to my brain­storm­ing process. Even so, it was still the same basic prin­ci­ple: get every­one in a room and throw out test ideas. The only dif­fer­ence was that, this time, we used data to iden­ti­fy prob­lems and the ideas we brain­stormed were about solu­tions. Now we were using data to make sure we were solv­ing real prob­lems. Unsur­pris­ing­ly, we saw much bet­ter results and could dri­ve real improve­ment. And because we were using data to iden­ti­fy cus­tomer pain points, we were also address­ing prob­lems our cus­tomers were hav­ing. Much bet­ter!

But some­thing was still miss­ing. Using analy­sis to iden­ti­fy prob­lems is great, but it will nev­er make you bet­ter. It will only help you fix what’s bro­ken. What hap­pens when you’ve run out of things that need fix­ing? I know, I know … nev­er should hap­pen, but at some point, the juice just might not be worth the squeeze. And if you use data to iden­ti­fy some­thing bro­ken, why test it when you could just fix it? I was think­ing these Jack Handey-esque deep thoughts about five years ago when it hit me: what if we flipped things around and used brain­storm­ing to iden­ti­fy prob­lems and data to sug­gest solu­tions?

In this new world sce­nario I had cre­at­ed, it would be easy for us to uti­lize exist­ing use cas­es or cre­ate new ones, and then run through the site expe­ri­ence from the per­spec­tive of those who would fit in those use cas­es. We could then iden­ti­fy areas of oppor­tu­ni­ty to make the site expe­ri­ence bet­ter for the cus­tomer (and more lucra­tive for the busi­ness). Once we iden­ti­fied areas to eval­u­ate, we could dive through the avail­able data (CRM, web behav­ioral, VoC, etc.) to devel­op poten­tial solu­tions to test. We were shift­ing our focus from fix­ing things that were bro­ken to tru­ly opti­miz­ing the cus­tomer rela­tion­ship with the brand.

It took just a few tri­als through this new process to under­stand its poten­tial and real­ize its ben­e­fit. But we’re in the busi­ness of con­tin­u­ous opti­miza­tion, right? So we can’t stop there! On a whim, I decid­ed to see what would hap­pen if I com­bined ele­ments of all of these method­olo­gies and I struck gold.

Let me break down what this looks like in prac­tice. On a reg­u­lar cadence, we meet for opti­miza­tion brain­storm­ing. Ana­lysts are asked to pre­pare data iden­ti­fy­ing poten­tial fric­tion points, busi­ness own­ers are asked to do a site run through using var­i­ous use cas­es to iden­ti­fy oppor­tu­ni­ties or issues. All come togeth­er to present their iden­ti­fied prob­lems and/or pro­posed solu­tions. Ana­lysts take all ideas and/or prob­lems cur­rent­ly with­out data sup­port as take­aways to con­duct the req­ui­site analy­sis to back up the solu­tion and/or pro­pose ideas for iden­ti­fied oppor­tu­ni­ties or prob­lems.

At the next reg­u­lar­ly sched­uled brain­storm­ing ses­sion, the ana­lysts come with what they learned about the iden­ti­fied ideas brought up in the pri­or meet­ing and busi­ness own­ers come with new areas to assess … and a cycle is born! Now, we get all the best ideas from all pos­si­ble areas and we use data both to iden­ti­fy prob­lems and oppor­tu­ni­ties! Plus, there’s the added ben­e­fit of teach­ing the entire orga­ni­za­tion to be data-dri­ven in their deci­sion-mak­ing. And the out­come? The most suc­cess­ful opti­miza­tion pro­gram I’ve ever been a part of to this day.

Want help with your opti­miza­tion ideation process?
Search Dis­cov­ery can help devel­op a new process or improve an exist­ing process from idea intake to pri­or­i­ti­za­tion. Hit us up!

Your Search Ends Here!