Love / Hate, A / B

I have a love / hate relationship with A / B tests. Leaning too hard on them to make design decisions can make for very anemic process. It encourages an incremental, guess-and-check approach that feels like a task better suited for an automaton. Even when isolating one variable, the results mainly speak to “what” had the effect on behavior, rather than the “why”. I’d rather be solving problems and taking bigger strokes. But you simply can’t argue with its place in the toolbelt, especially when seeing some of the results on ABTests.com

Dec 9 2009