The Friedman Test (presentation)
Abstract: 6 wine tasters taste 8 wines to be rated out of 10. How can we tell if any of the wines are outstandingly good (or bad)? One difficulty is that wine tasters don’t all rate on the same scale – Alice’s 2 is Bob’s 6 (Bob has a drinking problem). Another difficulty is that we don’t know how individual taster’s ratings are distributed – Alice rates everything either 2 or 9, Bob’s ratings are binomially distributed about 7.
The Friedman test is a nonlinear statistical test (on the rank sums of the ratings) designed to answer this type of question (caveat, caveat, caveat). This test was used to assess the output of a recent Navy expert elicitation session carried out by MCA in Sydney. In this session, I discuss the test and how we used it (I will not discuss the results of the assessment in order to keep the talk UNCLASSIFIED).
(Presented at the DSTO Mathematics Community of Practice Seminar.)