Being User Driven is a Tax

by Willem Maas on June 9, 2011

The folks at Aardvark.com have described in blog posts and presentations (like this one) how they combined user centered design and nimble engineering to create a hockey stick of engineering progress.

I think their product development process is mighty impressive and according to co-founder Max Ventilla their investors thought the same, giving “money as much for our process as for our team and concept.”

While going on gut instinct would have been “twice as fast,” Aardvark’s user driven process “dramatically reduced the chance that we would make wildly wrong bets and have to double back, abandoning large periods of work.”

Aardvark's product development process

Rather than going on gut instinct, they collected new ideas with a net cast widely, and mitigated risk by staging solutions in small chunks developed by small teams.

The product development process is detailed in posts here, here, and here on the Aardvark blog.

For me, 3  aspects of their process  stand out:

Qualitative/direct user research methods rather than quantitative/indirect sources like website analytics or A/B testing.  A few examples:

  • Every week a new group of 6-12 users would visit the office for prototype testing (paper, or click-through) or to be interviewed and observed using the product.
  • After testing a new feature on the live site with a small group of 10-20 users, each was sent an email containing 3-4 questions about their experience with it.
  • Phone interviews, a community forum, feedback sent via email, and “overheard” feedback on Twitter and Facebook.

Horizontal organization.  Each Friday the entire Aardvark team discussed the “Weekly Learnings” collected through research above.

  • Prioritized features or “learnings” were then solved by small groups of 3-4 people each — an engineer, a researcher, a designer, etc.
  • How learnings were prioritized isn’t clear.  Google, the founders’ alma mater, funds only the 4s and 5s on a five point ranking system.  That’s described here.

A deeply embedded evolutionary approach. Evolution progresses through small changes/adaptations and so did the “chunked” implementation of features at Aardvark.  Further, each “chunk” passing through the process was run through two think/make/check cycles  – the first three steps and the last three.

By the numbers:

  • Founded in July 2007.
  • Five concepts abandoned during prototyping.  The sixth idea, Aardvark, run as a Wizard of Oz experiment for 9 months during 2008 until publicly released as a minimum viable product.
  • Most compelling stat from October 2009 data: 87.7% of questions submitted were answered, and nearly 60% of them were answered within 10 minutes.
  • In February 2010 Google paid $50M to acquire the company.
If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.

Comments on this entry are closed.

Previous post:

Next post: