Using Quantitative Procedures Wisely

edited 8/19/11

Deborah Stone in Policy Paradox, (1998, New York: Norton) cautions the reader against assuming that political and ethical decisions can be avoided by the careful use of measurement techniques. Below are two sets of considerations which is it important to be apprised of.

1. Counting requires decisions about categorizing, about what (or whom) to include and exclude.

2. Measuring any phenomenon implicitly creates norms about how much is too little, too much, or just right.

3. Numbers can be ambiguous, and so leave room for political struggles to control their interpretation.

4. Numbers are used to tell stories, such as stories of decline ("we are approaching a crisis").

5. Numbers can create the illusion that a very complex and ambiguous phenomenon is simple, countable, and precisely defined.

6. Numbers can create political communities out of people who share some trait that has been counted.

7. Counting can aid negotiation and compromise, by making intangible qualities seem divisible.

8. Numbers, by seeming to be so precise, help bolster the authority of those who count.

1. People react to being counted or measured, and try to "look good" on the measure.

2. The process of counting something makes people notice it more, and record keeping stimulates reporting.

3. Counting can be used to stimulate public demands for change.

4. When measurement is explicitly used to evaluate performance, the people being evaluated try to manipulate their "scores."

5. The power to measure is the power to control. Measurers have a lot of discretion in their choice of what and how to measure.

6. Measuring creates alliances between the measurers and the measured.

7. Numbers don't speak for themselves, and people try to control how others will interpret numbers.

Get a copy of Stone's book and read it. It is well worth the time and effort. EGR

P.S. See also Rozycki, E (2014)Measurability and Educational Concerns