Report

WINETASTER ON 11/07/11 WITH 8 JUDGES AND 8 WINES BASED ON RANKS, IDENT=N
Copyright (c) 1995-2011 Richard E. Quandt, V. 1.65

FLIGHT 1:
Number of Judges = 8
Number of Wines = 8

Identification of the Wine: The judges' overall ranking:

Wine A is Chambertin Clos de Bèze 1990 P. Gelin ........ 5th place
Wine B is Clos de la Roche 2003 Girardin ........ 2nd place
Wine C is Clos de la Roche 1995 Ponsot ........ 8th place
Wine D is Chambertin Clos de Bèze 2003 Drouhin ........ 1st place
Wine E is Clos de la Roche 1999 Amiot ........ 4th place
Wine F is Chambertin Clos de Bèze 1995 Jadot ........ 3rd place
Wine G is Clos de la Roche 1990 Dujac ........ 6th place
Wine H is Chambertin Clos de Bèze 1999 Faiveley ........ 7th place

The Judges's Rankings

Judge Wine -> A B C D E F G H

Alexa 6. 4. 8. 3. 1. 2. 5. 7.
Zaki 4. 3. 5. 2. 6. 1. 8. 7.
Orley 7. 8. 5. 4. 6. 1. 2. 3.
Burt 2. 3. 8. 1. 4. 5. 6. 7.
Bob 1. 3. 8. 2. 5. 6. 7. 4.
Ed 7. 1. 8. 2. 3. 5. 6. 4.
Mike 7. 2. 3. 4. 5. 6. 1. 8.
Dick 3. 2. 5. 1. 4. 6. 8. 7.

Table of Votes Against

Wine -> A B C D E F G H

Group Ranking -> 5 2 8 1 4 3 6 7
Votes Against -> 37 26 50 19 34 32 43 47

( 8 is the best possible, 64 is the worst)

Here is a measure of the correlation in the preferences of the judges which
ranges between 1.0 (perfect correlation) and 0.0 (no correlation):

W = 0.2887

The probability that random chance could be responsible for this correlation
is quite small, 0.0236. Most analysts would say that unless this
probability is less than 0.1, the judges' preferences are not strongly
related.
We now analyze how each taster's preferences are correlated with the group
preference. A correlation of 1.0 means that the taster's preferences are a
perfect predictor of the group's preferences. A 0.0 means no correlation,
while a -1.0 means that the taster has the reverse ranking of the group.
This is measured by the correlation R.

Correlation Between the Ranks of
Each Person With the Average Ranking of Others

Name of Person Correlation R

Burt 0.8333
Dick 0.6905
Alexa 0.6429
Zaki 0.5952
Ed 0.5714
Bob 0.3832
Mike -0.0476
Orley -0.4286

The wines were preferred by the judges in the following order. When the
preferences of the judges are strong enough to permit meaningful differentiation
among the wines, they are separated by -------------------- and are judged to be
significantly different.

1. ........ 1st place Wine D is Chambertin Clos de Bèze 2003 Drouhin
---------------------------------------------------
2. ........ 2nd place Wine B is Clos de la Roche 2003 Girardin
3. ........ 3rd place Wine F is Chambertin Clos de Bèze 1995 Jadot
4. ........ 4th place Wine E is Clos de la Roche 1999 Amiot
5. ........ 5th place Wine A is Chambertin Clos de Bèze 1990 P. Gelin
6. ........ 6th place Wine G is Clos de la Roche 1990 Dujac
---------------------------------------------------
7. ........ 7th place Wine H is Chambertin Clos de Bèze 1999 Faivelet
8. ........ 8th place Wine C is Clos de la Roche 1995 Ponsot

We now test whether the ranksums AS A WHOLE provide a significant ordering.
The Friedman Chi-square value is 16.1667. The probability that this could
happen by chance is 0.0236

We now undertake a more detailed examination of the pair-wise rank correla-
tions that exist between pairs of judges. First, we present a table in which you
can find the correlation for any pair of judges, by finding one of the names in the
left hand margin and the other name on top of a column. A second table arranges
these correlations in descending order and marks which is significantly positive
significantly negative, or not significant. This may allow you to find clusters
of judges whose rankings were particularly similar or particularly dissimilar.

Pairwise Rank Correlations

Correlations must exceed in absolute value 0.74 for significance at the 0.05
level and must exceed 0.64 for significance at the 0.1 level

Alexa Zaki Orley

Alexa 1.000 0.405 0.071
Zaki 0.405 1.000 -0.071
Orley 0.071 -0.071 1.000
Burt 0.524 0.548 -0.429
Bob 0.143 0.357 -0.500
Ed 0.595 0.286 -0.238
Mike 0.048 -0.119 -0.095
Dick 0.286 0.619 -0.690

Burt Bob Ed

Alexa 0.524 0.143 0.595
Zaki 0.548 0.357 0.286
Orley -0.429 -0.500 -0.238
Burt 1.000 0.833 0.524
Bob 0.833 1.000 0.452
Ed 0.524 0.452 1.000
Mike -0.048 -0.405 0.095
Dick 0.810 0.690 0.500

Mike Dick

Alexa 0.048 0.286
Zaki -0.119 0.619
Orley -0.095 -0.690
Burt -0.048 0.810
Bob -0.405 0.690
Ed 0.095 0.500
Mike 1.000 0.048
Dick 0.048 1.000

Pairwise correlations in descending order

0.833 Burt and Bob Significantly positive
0.810 Burt and Dick Significantly positive
0.690 Bob and Dick Significantly positive
0.619 Zaki and Dick Not significant
0.595 Alexa and Ed Not significant
0.548 Zaki and Burt Not significant
0.524 Burt and Ed Not significant
0.524 Alexa and Burt Not significant
0.500 Ed and Dick Not significant
0.452 Bob and Ed Not significant
0.405 Alexa and Zaki Not significant
0.357 Zaki and Bob Not significant
0.286 Alexa and Dick Not significant
0.286 Zaki and Ed Not significant
0.143 Alexa and Bob Not significant
0.095 Ed and Mike Not significant
0.071 Alexa and Orley Not significant
0.048 Mike and Dick Not significant
0.048 Alexa and Mike Not significant
-0.048 Burt and Mike Not significant
-0.071 Zaki and Orley Not significant
-0.095 Orley and Mike Not significant
-0.119 Zaki and Mike Not significant
-0.238 Orley and Ed Not significant
-0.405 Bob and Mike Not significant
-0.429 Orley and Burt Not significant
-0.500 Orley and Bob Not significant
-0.690 Orley and Dick Significantly negative

COMMENT:

The wines as whole were tremendous and the differences were secondary to
the overall pleasure that the group had from drinking them. There were
few if any full or partially full glasses left on the table. Having said all that,
2003 was a clear winner in the tasting in both appellations. The design
of this winetasting was to permit comparison of vineyards and vintages.We first
note that the overall agreement in the group was very strong. The rank sums themselves
can be summed in two ways: over vintages and over appellations. Doing this the first way,
we get rank sums of 80, 82, 81 and 45 for 1990, 1995, 1999 and 2003 respectively. Doing it the other
way we get 153 for Clos de la Roche and 135 for Chambertin. While the difference in the
ranksums over appellations is not significant (although Chambertin edges out Clos de la Roche),
and while the sum of rank sums for 1990, 1995 and 1999 are essentially identical, 2003 is
rated significantly better than the other three vintages (See Richard E. Quandt, The Journal
of Wine Economics, May 2007, pp. 98-102), which is further interesting because the 2003
wines were marginally less expensive than the others. What is further convincing evidence that
the vintage effects are much more important than the appelllations effects is the the best two
wines include a Clos de la Roche and a Chambertin, while the worst two wines also include one of
each appellation.

Overall the vintages to trump the vineyards. The group felt that 2003s were a
surprise.

Return to previous page