Jim Lewis (a usability practitioner with IBM for over 20 years - here are some of his publications) has an article on Sample Sizes for Usability Tests in the November-December 2006 issue of <interactions>. His article is part of the special edition on "Waits & Measures" (aka "measuring usability").
Do not let the subtitle of the article scare you ("Mostly math, not magic") - the math is not that difficult to follow. I expect to see replies to Jim's article from several places. User Interface Engineering (note that Lori Landesman, one of those UIE authors in 2001, is now an IBMer too). And Jakob. And PeterMe / Dennis Wixon to remind us that asking "how many users" is pointless.
The other article in that issue by an IBMer is the regular column "Pushing the Envelope" by Fred Sampson. Fred sets the stage for a future issue about the user experience challenges with Web applications / Web 2.0 / social computing (or something like that) - Whither the Web?. This will also be our internal-IBM topic for World Usability Day on Tuesday.
Comments
Errors in <interactions> article
FYI, I was pleased to see the article in interactions, but noticed problems in two formulas. It looks like the typesetter made a mistake in the first formula (p. 30), failing to superscript the n. On p. 31, there's an extra symbol (>) in my formula. For correct formulas and a more complete treatment, feel free to ask me for a copy of Lewis, J. R. (2006). Usability testing. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (pp. 1275-1316). New York, NY: John Wiley.
Useful Perspective
I enjoyed Jim's article and appreciated getting this mathematical perspective on the question of "How many users is enough?" Ever since Christine Perfetti and I wrote our article on the topic a few years ago, we've gotten a lot of response, but it's always been simple agreement or disagreement with us. Jim's piece was the first I've seen that came at the problem from a different perspective and shed some new light on the issues we've been grappling with for a long time. I'm looking forward to diving into the formuli and understanding this more. Thanks for pointing out the errors in the text, Jim.