Tests Prove It: Real Life Outperforms Intuition

by Willem Maas on August 3, 2011

Visual Website Optimizer’s interview (here) with direct response copywriter Jeremy Reeves is a dollars-and-cents example of how experimentation (in this case A/B testing) can yield better results than intuition.

I’d have thought short-form digital content and pithy calls-to-action would outperform longer variations, prior to reading Jeremy’s interview.

  • That’s been my own experience with usability testing, as well as my own browsing/reading behavior online.
  • And it also seems to be the overall trend in digital: Twitter and Youtube being two standout examples.

Yet real life is more complex.  One long-form sales letter (all 50 pages of it here!) Jeremy developed for a client outperforms by 50% its short predecessor (a video, a few bullets, summary of the offer, etc) .

Personally, I’m skeptical.  This sales letter evokes too many childhood memories of ads for Veg-O-Matics and Ginsu knives on late-night TV.

But I admire and respect the data-driven savvy of direct marketers like Lester Wunderman who built the market for performance-based advertising and paved the way for Google.

2 Stars Apiece: Many Unhappy Users of WSJ and NYT Mobile Apps

July 8, 2011

Jakob Nielsen wrote a great piece this week (here) analyzing the miserable ratings reviewers have given the Wall Street Journal’s iPhone app.  Across 68,418 reviews, the WSJ’s app gets an overall 2 star rating. Nielsen attributes this to a confusing interface design.  ”It’s clear that people are deeply offended by being asked to pay again for mobile [...]

Read the full article →

Being User Driven is a Tax

June 9, 2011

The folks at Aardvark.com have described in blog posts and presentations (like this one) how they combined user centered design and nimble engineering to create a hockey stick of engineering progress. I think their product development process is mighty impressive and according to co-founder Max Ventilla their investors thought the same, giving “money as much for [...]

Read the full article →

How TaskRabbit’s Redesign Nearly Paid for Itself with A/B Testing

May 27, 2011

Janice Fraser describes how TaskRabbit earned an extra $26K in revenue by launching a redesigned homepage 30 days before the design was “pixel perfect” and completely integrated throughout the site.  A/B testing the redesign on the homepage only, they found a 9.75% “task posted” conversion, where the existing design converted at 5%. Watch live video from [...]

Read the full article →