Reply to "Grading Measurers?"

I analyzed the Pete’s data to help provide a bit more statistical description of “experience" as a factor to consider in determining whether -- and, if so, what kind of -- a scale should be used in "grading" (loaded term) or classifying measurers in response to requests to assess qualifications of measurers similar to that currently in place for officials.

Pete’s sample includes 443 “active” measurers, that is, persons who measured at least one course in the past 35 months. The sample has some duplication due to typographical errors in measurers' last names, but not enough to substantially influence the results of this analysis.

In 2004, 111 measurers, or 25.1% of active measurers, measured at least one course.

In 2005, 189 (42.7%) measured one or more courses for certification.

Thru Nov. 2006, 238 (53.7%) have measured at least one course.

The overall productivity of this sample was examined in terms of a 3-year average. The mean annual number of courses per measurer was 2.99, with a median of 1.0. The average annual minimum number of courses was 0.33 (the “active” threshold). The average annual maximum was 38.67. Slightly more than 22% (344) measured 3 or fewer courses per year and less than 15% (377) of the sample measured 5 or fewer courses per year.

The following chart displays the 3-year average number of course measurement certifications by the number of measurers. Not unexpectedly, the distribution is highly skewed toward a small number of active measurers. Approximately 16% of active measurers accounted for two-thirds of the 3,976 course certifications completed in the 35-month period.



I don’t draw any conclusions at this time about the usefulness of these results for “classifying” measures, preferring to leave that to further discussion. And, of course, this analysis does not shed any light whatsoever on the subjective “quality” of the work, but simply the more objective, quantitative aspects of measurer productivity. One item of note, however, is that a substantial number of course certifications are performed by individuals who do not often engage in this activity. The contribution of these “low-producers” to overall productivity of course certification should not be ignored. As a counterpart to the small group of high-producing active measurers, this group supports the principle that course measurement can be (and is) done by a broad spectrum, and is not limited to a select few.