 Why are the experimental results somewhat irrelevant?
The members of the team, as noted in the original article, all hold day jobs at Microsoft, mainly as designers, managers, and coders for Microsoft Word. The point of the second test was not to find the difference between cheap and expensive beers but instead to compare a variety of top-of-the-line beers. Could they detect any of the subtleties of brewing style and provenance that microbrew customers pay such attention to when choosing some Doppelbock over a cream ale? Since the tasting panel had left the first round grumbling that cheap lagers were not a fair test of their abilities, this second round of testing was advertised to the panel as a reward. One aspect of the reward was that they would presumably enjoy the actual testing more--fewer rueful beer descriptions along the lines of "urine" or "get it away!" were expected than in the first round. The other aspect of anticipated reward was the panelists' unspoken but obvious assumption that this time they would "do better" on the test. Intellectual vanity being what it is, people who had fought for and won jobs at Microsoft and who still must fight every six months for primacy on the employee-ranking scale (which determines--gasp!--how many new stock options they receive) would assume that their skill as tasters was on trial, just as much as the beer was. Here is what happened and what it meant: 1. Procedure. This was similar in most ways to the experimental approach of Round 1. The nine testers who showed up were a subset of the original 12. The missing three dropped out with excuses of "my wife is sick" (one person) and "meeting is running long" (two). As before, each tester found before him on a table 10 red plastic cups, labeled A through J. The A-to-J labeling scheme was the same for all testers. Descriptions of and comments about each beer's taste--"smooth and nutty," "too strong," etc. They were asked to put each beer in its proper category--and to name a specific brewery and brand if they could. The idea here was to test the veteran beer drinkers' claim to recognize the distinctive tastes of famous brands. 2. Philosophy. This is in contrast with an ale, which is brewed faster, warmer, and with the yeast on top. The first test was designed to evaluate low-end beers and therefore had to be lager-centric.
 Part of what matters is the label itself
 The experimenters were unqualified  The experiment subjects were unqualified  The sample size was too small  Part of what matters is the label itself