Reply #3: That is one of a world of non-parametric statistical tests...different issue. [View All]

K-S is hypothesis testing of distributions, and there are also non-parametric statistics for correlation, hypothesis testing, and many other applications.

In this case, the notion is that the quality of the original data usually reported descriptively as "meaningful" (grades, etc.). Raw scores in this case don't measure up (pun intended). If you determined that the grade point average at Harvard (as a distribution) was NOT "significantly" different than the grade point average at Podunk Junior College; which is a likely finding - would you conclude that a B average at Podunk was the equivalent of Harvard?

The original grade point average data is not compared on the same meaningful metric of knowledge, so the conclusion is flawed.

If you apply a statistical test to an hypothesis, you'd be responsible to meet the mathematical assumptions of the test and be aware of the limitations of the data that you were using...also something that many authors don't do! Supposedly, scholarly journal reviewers check for such things even though you can find glaring examples of problems in the most prestigious scientific publications.

Studies of published research indicate that the quality of the original data is one of the most common problems. GIGO (garbage in, garbage out). It is relatively easy to get interval level (good quality) data for "grading", but most schools don't even try.