Confidence Intervals and Usability Testing

Measuring the User Experience, by Tom Tullis and Bill Albert, talks about the use of “Confidence Intervals” within usability testing. The authors describe Confidence Intervals as “a range that estimates the true population value for a statistic.” It comes across as a better way of obtaining more accurate testing results without having to greatly increase the number of test participants. As user interface and user experience designers, we may tend to feel that we can detect a pattern early on in our usability testing among potential users, and seek to address and fix the perceived problems without the need for more testing. Confidence Intervals create a way to find out if our perceptions are indeed accurate before more time and resources are committed to solving a problem that may not exist.