Testing methodology- how do you prioritize?

Sonny Forelli

Sonny Forelli

Well-known member
May 8, 2008
2,330
89
48
Liberty City
I've thought of this for ages, and think I"m finally going to get more regimented in my approach. Yes I'm aware (and often preach on here) about statistical significance, not making rash decisions, dayparting, etc.

But I'm wondering how you handle really broad testing. That is- let's say you're launching a mortgage campaign (random example).

How do you handle the following.

Background colors. Stock photo nationalities, Audio/video walk on vs not. 1 page form vs 2 vs 3. Headline call to action/text.

That is- how do you start? I usually am inclined to build two distinct versions of something, keep everything else the same (that is- 1 page vs 3 page in this type of example), run those. If clear differences pick that and then move on to following things- headlines, background colors, etc.

However this is all very crude and while data is significant before making any calls and then I do things like weighting in new revisions at 10-20% I still feel like I"m very unorganized here.
 


There is no specific formula to split testing. You've gotta test all variables in combos and you're bound to find a winner. Get messy if you have to, during setup; in the end if you're tracking carefully you'll deter your performance.
 
GrindHard said:
There is no specific formula to split testing. You've gotta test all variables in combos and you're bound to find a winner. Get messy if you have to, during setup; in the end if you're tracking carefully you'll deter your performance.
Click to expand...
'

yeah, I have a feeling a lot of others (frankly a lot of large companies) just adopt this approach.

I've long wondered if plugging into some company like a rubicon (though not the greatest example in this case - but someone w/ a solid algorithm) and building pages to be able to toggle in variables on the back end would be smart. IE - mesh w/ some company's testing logic so that their page would either a) load in a million versions OR b) <what I want> be able to build my page so that their system could dynamically control all of my variables so that it could - much better/faster/automatically using much better best practices narrow down the winning formula.
 
Sonny Forelli said:
'

yeah, I have a feeling a lot of others (frankly a lot of large companies) just adopt this approach.

I've long wondered if plugging into some company like a rubicon (though not the greatest example in this case - but someone w/ a solid algorithm) and building pages to be able to toggle in variables on the back end would be smart. IE - mesh w/ some company's testing logic so that their page would either a) load in a million versions OR b) <what I want> be able to build my page so that their system could dynamically control all of my variables so that it could - much better/faster/automatically using much better best practices narrow down the winning formula.
Click to expand...

Sounds like you need another human :D - Outsource the grunt work if you've got the money, but don't dwell on an automated solution because there's a lot experience you build in setting all of this up and that knowledge is the best. Helps you get analytical and monitor your expenses more efficiently.
 
good question. i always think to myself that there is a better approach to testing than i am currently using. would love to hear from some veterans about this.
 
People hit your lp, here is what I think they will see in the following order

  1. Template, color and layout
  2. Article / LP title, headline
  3. Pictures on lp if any
  4. Content
  5. Call to action
So, that's my split test order except call to action. I put call to action before content because a lot of people they click without reading the whole thing.
Forgot to mention, if there is video/audio on the landing page, I split test them before the picture.
 
Jardini said:
People hit your lp, here is what I think they will see in the following order

  1. Template, color and layout
  2. Article / LP title, headline
  3. Pictures on lp if any
  4. Content
  5. Call to action
So, that's my split test order except call to action. I put call to action before content because a lot of people they click without reading the whole thing.
Forgot to mention, if there is video/audio on the landing page, I split test them before the picture.
Click to expand...

that's a very good point, thank you
 
I like to start with a macro test. Create 2 variations of your control. Change the overall message/theme of each one. Split test headline/copy/image. Don't worry about small things like bg. color.

After u have a winner of the 3, you can then do a micro test where you stay in the same theme, but play with wording, or just image, maybe background color. Then fine tune. Throw in other macro tests from time to time.
 
You must log in or register to reply here.
Top Bottom