An earlier version of this article was published in Evelyn M. Powers & James McKernan (eds.)
Philosophy and the Educator's Quest for Vision Yearbook of the Forty-Second Annual Meeting of the South Atlantic Philosophy of Education Society.
(Greenville, NC: East Carolina University School of Education, 1998) pp.106-117.

RETURN
updated 5/9/19

Efficiency assessments ... provide a frame of reference for relating costs to program results.
---- Rossi and Freeman (1)

Thomas Gradgind. With a rule and a pair of scales,...ready to weigh and measure any parcel of human nature and tell you exactly what it comes to.---- Charles Dickens, Hard Times

CONTENTS

Introduction

On what basis should we judge instructional methods? What is better teaching? Why are some materials to be preferred to others? In this paper I propose a heuristic for obtaining reasonable, practical answers to these questions. I will take the central question of our enterprise to be this: What makes one method better than another? This question is central because we can answer questions about better teaching and better materials in terms of it. For example, some kind of teaching is better than others inasmuch as it incorporates better methods. Some materials are better than others inasmuch as they can be better employed in a preferred method. Thus to answer the question, "What makes one method better than another?" is indirectly to answer questions as to what is better teaching and what are better materials.

On the face of it, this seems to counter common experience: Surely someone could teach badly despite using a superior method -- a powerful car does not always make for a fast driver. True. Teachers are individuals and some individuals seem to succeed at teaching no matter what method they employ; others can't make out with anything. I would not deny that individual gifts, personality, can contribute -- and interfere -- much in the learning process. But if the concept of method has any validity whatsoever then it must mean something which can remain constant in teaching interactions when teachers, students and physical environs vary.

This is an important conceptual point. If we could establish, for example, that student achievement, correlated with, say, teacher-personality-type and nothing else, it would spell the end of instructional methodology. Psychological pretesting would select those who teach best and we wouldn't need to be bothered with techniques, materials, physical plant and the like. But perhaps -- you interject -- even good teachers teach better when properly trained. If this is true -- and it probably is -- it contradicts the idea that only teacher-personality-type correlates with student achievement.

Before we continue, however, there are few points that bear mentioning. The first is that goals or objectives may be substantially influenced by method. What it is that one learns in learning, say, Spanish, might be at any time in the student's development quite different depending upon whether one uses a grammar-translation method, or an audio-lingual method. But the vagueness of the desired outcome, "knowing Spanish," often obscures this.

The second is that some learnings are not causally related to teaching interventions. Much of what character education pursues is what H.L. A. Hart has called defeasible ascriptions (2). No criterion of sufficiency exists that logically compels their bestowal. Honesty, for example, is a characteristic not learned but conceded. This is why rehabilitation is necessarily far less than a "scientific" enterprise (3) The perennial controversy over the primary goal of university education, i.e. the edification of undergraduates versus the development of technical skills in them, rests on this confusion (4).

There are, however, other areas in which the distinctions among, goals, methods, teacher skill, etc. can be reasonably made out. We will restrict our attention accordingly.

Method

What is a method? That is a difficult question to answer in any positive manner, e.g. with a definition. We will leave it until after our main pursuit is exhausted. Suffice it for now to note what "method" is generally not intended to denote; for example: teacher or student personality-type, sexual differences, student or teacher socio-economic status, size of the school population or physical plant, etc.

Few methods in education are fully specified as to the circumstances in which they can be expected to work; rather, what passes for method is not infrequently an amalgam of a little research, a lot of anecdote and more conjecture brought to life by the enthusiasm of a teacher comfortable in its use. Ideally, -- and we will be satisfied with less than that -- a method should indicate those circumstances in which it is applicable and what variables in the teaching interaction influence its effectiveness. One need only review any of a number of texts on classroom management, for example, to realize that this criterion is seldom met. (5) (This ideal situation is not unlike the pharmacological information given along with prescription medicine, e.g. dosage, purpose, effects, counterindications.)

The very concept of method presumes that the distinction between circumstantial and methodological variation is a viable one. But it is a matter of fact whether such a distinction obtains in the real world. A methodologist's intents do not guarantee that a procedure will be an effective method.

We find ourselves at this point: we believe that there are such things as educational methods, although we might find out that specific procedures are of little, if any, effectiveness. There may be little consistent research to back up our beliefs. Yet the very enterprise of teacher training rests on our as yet generally scientifically unsubstantiated faith in the existence of educational method. I will not argue to undermine this faith: I am a somewhat of a believer myself. Rather, I will lay out the tenets of this faith in their complexity to show why questions of method evaluation are often controversial and difficult to answer. We shall see that the complexity of methodological decision in education reflects directly the complexity of the institutional contexts in which education is carried out.

Means and Ends

A method is, in the very least, a means to an end. Certain methods may be delightful in their application but it is in terms of their ends that they are more likely judged.

As means, one method is better than another if it is more efficient, more cost-effective, in achieving its ends; that is, if it achieves a sought-after effect for the lesser expenditure in time, money or effort. This is standard doctrine.

However, it is not in defining "better" or "efficient" that controversy is broached in trying to evaluate methods. Efficiency is, by far, the least important of the considerations that are brought to bear in real-world assessments of method. A variety of other interests are involved in deciding the desirability of a proposed procedure. We will examine below some of those interests so as to gain an understanding of how methods are judged in accord with them.

Playing an important role in assessing methods are:

a. moral interests;

b. educational "status" interests;

c. student interests;

d. teacher interests; and very importantly,

e. institutional interests.

Pedagogical Efficiency

Preliminary to our examining consideration that bear on the evaluation of methods, we will define a "baseline" for the purposes of comparison. That basis of comparison will be the concept of pedagogical efficiency.

Not all human behavior can be enhanced by teaching interventions. From all possible human behavior, the scope of method is restricted to that behavior that can be enhanced by teaching interventions. Let us call this subset "type-T behavior." (See the Venn diagram in figure 1a.)

Because we do not believe that teaching interventions could enhance physical growth in most ways -- body-building may be an exception to this -- we do not have "Methods for Tallness Education" or "Techniques for Sleep Class." (See chart 1b.)
 

Figure 1a.

Chart 1b.

But we do talk of methods of dieting because we believe that, in general, one's body weight can be affected through some human intervention. Obesity, in fact, may not be controllable for many people, short of complete starvation. Were this impossibility of weight control to become generally accepted, we would abandon talk of dieting methods for such persons not affected by them.

A method is more or less pedagogically efficient to the extent that it requires less or more, respectively human resources to enhance, by teaching interventions, behavior of type T.

The practical problem of the evaluation of method is basically this: given that a procedure is acceptable or even desirable according to some interest, e.g. kids like to watch TV, it is possible that as a procedure, e.g. TV presentation of material, it is not pedagogically efficient to any degree, i.e. it has no effect on human behavior of type T. (It is a matter of taste, perhaps, or of polemic strategy whether we say of a procedure that it is a method of no pedagogical efficiency or that it is not a method, at all.)

Large group lecture may be very desirable from an institutional point of view because it maximizes the student-to-teacher ratio and thus the tuition-to-salary ratio. But large group lecture may often be pedagogically inefficient for certain purposes. Consider, for example, that the cash value of a high IQ may be that one is teachable by methods -- large group lecture -- that are pedagogically of low efficiency for people of lower IQ's. Institutionally, high IQ is valued for its cost-efficiency.(6) Certain human traits are valued because of the social circumstances (7) in which learning takes place, not because they enhance some absolute of pedagogical efficiency (8).

Method and Moral Interests

Contrary to much common opinion about education, I state here with utmost conviction: there is no "motivation problem." There just isn't. We know how to motivate people; we've known from time immemorial. It is just that there are other considerations that inhibit effectiveness in the matter of motivation. We know what, among other things, motivates people to work: e.g. money, sex, fear. If one wants to maximize pedagogical efficiency in a school, use real money bonuses (a standard practice in many businesses), a brothel and a torture chamber. And just watch those test scores rise!

Clearly, it is not a lack of knowledge of motivators that curtails our methodology. Rather, there are ethical limits on what we may be willing to employ. Figure 2a. illustrates an important consideration: not every efficient method may be moral.

Chart 2b. gives some examples of classes of procedure in terms of efficiency versus morality.
 

Figure 2a.

Chart 2b.

Chart 2b. is offered for purposes of illustration only. What is moral or not, and under what circumstances, is usually a controversial matter. Some people liken corporal punishment to torture and others regularly employ sexual stimulation ("flirting," etc.) as a teaching technique. But the point is clear: not every efficient method need be moral; nor, every moral method, efficient.

The more diverse the population a school district comprises, the more dissension we might expect to find about what is an acceptable educational method. In public schools, for example, students can be excused "for religious reasons" from participation in gym and from saluting the flag. Is this effective physical or citizenship education? Most likely, not; but it illustrates an earlier claim that pedagogical efficiency is not the highest of priorities.

Method and Status

Not every T-type behavior, i.e. behavior enhanceable by teaching intervention, is a candidate for the school curriculum. This is so, even if such behavior is generally morally acceptable. For example, consider such things as telephone-book-ripping, hopscotch-playing or fireplug-vaulting. Such activities lack sufficient status to be admitted to the school curriculum.

The curriculum is composed of methods for enhancing behavior thought to have either intrinsic value, e.g. speaking certain English dialects, or instrumental value of appropriate status, for example, cooking, even auto repair, perhaps, but not cleaning toilets, trash-collecting or slaughtering cattle. Thus the range of acceptable effective method is cut even more. Figure 3a. shows the even more restricted range of efficient methodology. Chart 3b give some examples of various procedures.
 

Figure 3a.

Chart 3b.

This brings us to another banal but important point: the moral value or social status accorded a method is no guarantee of its pedagogical efficiency.

Method, Student and Teacher Interests

It is important to distinguish from the outset between what is wanted by students and what is wanted by others for the sake of students; the term "student interests" is ambiguous enough to be used for both of these. Because, for example, students want good grades, some of them find cheating a method for obtaining good grades. Others study for the exams and soon forget most of what they have studied.

If others can presume to know what is best for students -- and as teachers, we all do and must do so -- they select methods to counter cheating and cramming. The important consideration here is this: what a student sees as in his or her interest may not be what others see as such, and vice versa. Furthermore, there is no necessary connection between student interests of either kind and what may be of status educationally, morally acceptable, or pedagogically effective. Figure 4a. illustrates the contingency of types of student interests and figure 4b. illustrates the contingency of interests with status, morality and efficiency.
 

Figure 4a.

Figure 4b.

But what about student needs? I have deliberately chosen to avoid using the term "needs" and have used "interests" instead. This is because "needs" is an emotionally overladen, presumptuous term.(9) To call something a need is a way of insisting that it ought to be attended to, or that it is a means to something that ought to be attended to.

Do plants need water? For what? To survive? Yes, but we need not water them unless we agree they should survive. Do weeds need water? One person's clover is another's weed.

What do students need? This question is only a more emphatic way of asking what we want for the sake of students. If we lack consensus on this latter question, we are not going to attain it by reformulating it in terms of needs. It is, of course, important to distinguish between whimsies, wishes, likes, hankerings and wants. But we do not obtain a more objective conception of student interests (or anyone else's interests, for that matter) by begging the question of consensus by invoking a student's needs.

Parallel comments can be made about teacher interests: there is a difference between what teachers want and what others want for their sake. These different kinds of wants are logically independent of each other and also of student interests of both kinds and logically independent, too, of moral, status and efficiency interests. And it does not clarify to talk of teacher needs, either.

Some example of teacher interests generally not shared by students are: appreciation of subject matter, coherence of curriculum, desire that the class as a whole may do well, desire for professional advancement, desire for quieter or smaller classes, etc. One particular concern of teachers merits special comment. I call it the Principle of Triage from the battlefield medical practice of dividing the wounded into three groups so as to maximize the benefits of scarce medical supplies.

A casualty is either beyond the hope of recovery, so medicine is not wasted on him, or not hurt enough to receive the scarce medical treatment. Or, if none of these, he is a prime candidate. There is a very basic principle of rationality at work here: one attempts to optimize effect so as to avoid the charge of having allowed someone to die who might have been saved, by wasting scarce resources on either the doomed or the lucky.

A very similar principle is invoked in most educational situations. For a given method there are those who will not learn from it in the time allotted. There are those who will learn the "subject matter" whether or not the method is used with skill (or despite it -- the so-called "gifted.") And there are those who will gain the most from the teacher's giving them most of his time. Thus the Principle of Triage becomes, in education, the Practice of Teaching to the Middle. The general principle is, again, the Maximization of Beneficial Effects, or call it Optimization of Expenditure of Effort.

Special Education is an example of the abandonment of this principle of optimization which is, in effect, in conflict with certain ethical principles of opportunity. The courts have stepped in and prescribed methods of dealing with "special" children that have very tenuous relationships, if any, to concerns for pedagogical efficiency. So it goes in the real world of the schools. It is an open question whether any method which would be acceptable in terms of all the interests mentioned in this essay, moral, status, student, teacher and institution, would have any hope of also being pedagogically efficient.

Institutional Interests

By institutional interests I mean to indicate any additional demands placed in the teaching interaction by virtue of the fact that it takes place in a particular context, for example, a school, a church, a one-room-schoolhouse, or a large comprehensive high school. Some common institutional interests that place severe strains on many teaching methods are the demands for grading, attendance-taking, teaching by a bell-schedule, material inventory-taking, loudspeaker interruptions, etc. The more bureaucratized the school, the greater will be the bias towards pedagogical methods which facilitate supervision and evaluation, e.g. learning packets, levels-approaches, multiple-choice tests, or piecemeal learning objectives. Teachers will be expected to make detailed lesson plans (to be left for substitutes, perhaps) on the apparent theory that personnel are interchangeable and that process variables are completely under the teacher's control.

An interesting example of begrudged recognition of the conflict between institutional interests and pedagogical efficiency is found in so-called "scholastic achievement" tests. These tests can be easily administered to large groups of students; they are multiple choice and easily scored. But the following obvious considerations seem to escape many: no one can presume that a score on such a test measures the scholastic achievement of a given individual unless one has reason to believe all of the following:

a. the person who took the test is the person identified as the testee.

b. the mechanical procedures for scoring functioned well.

c. the test has established internal and external validity.

d. the testee has not cheated or been coached to the test.

e. the testee has made a "decent" effort to take the test.

f. those skills sampled by the test have been uniquely developed in school

What is interesting is the euphemism "scholastic achievement test" when what we have at most is a sampling of skills useful in school. One can't tell from the score where the student learned the items! (Imagine the California Achievement Tests renamed The California Sampler of Scholastically Relevant Skills -- it loses a lot of its cachet.)

Item f., that the skills sample have been learned in school, is a serious consideration. If those skills and knowledge that most contribute to success in school are acquired successfully outside school -- as researchers have long claimed (10) -- then "test of scholastic achievement" is a grim euphemism, indeed.

It is clear that institutional interests need have nothing to do with promoting pedagogical efficiency, or any other of the interests we have reviewed. No one who has ever worked in a large school system doubts which interests, pedagogical efficiency or administrative convenience, for example, takes precedence when push comes to shove.

To be an administrator and, at the same time, an educator has got to be a hard job. One must address the demands of various interests so as to survive in one's job, yet in such a way as to maximize pedagogical efficiency.

The thrust of these considerations has been to emphasize the possibilities of conflict among the various interests that constitute an educational enterprise. No doubt this possibility of conflict casts a somewhat pessimistic pallor over the whole argument. But perhaps a reasonable pessimism is appropriate (11). Educational problems are not the more easily attacked for their being underestimated in their ramifications.

Some More on Pedagogical Efficiency

I have used the concept of pedagogical efficiency as a basis of comparison for such notions as moral interest, student interest, etc. A method is pedagogically efficient to the extent that it enhances T-type behavior, that is, behavior than can be changed by teaching interventions. This concept of pedagogical efficiency is far from being "scientifically neutral." I will define a value neutral concept to compare it with so that the bias is obvious. The ramifications of this bias will be discussed.

Let us call a method "behaviorally effective" if it brings about some change in human behavior. Clearly anything that is pedagogically efficient is behaviorally effective, but not vice versa. For example, lobotomy might be behaviorally effective in that it transforms an intelligent speaker into a babbler. Such a lobotomy, thus, is not pedagogically efficient, because it would not be an enhancement. Although one might be able to be trained to mimic a babbling idiot -- an acting skill for certain comic parts, say-- becoming a real babbling idiot is quite a different matter.

Clearly, the bias in the concept of pedagogical efficiency has to do with the notion of enhancement. To enhance is to make better in some respect, often in some subtle way. Therefore, unless there is agreement as to what it is to be made better, the concept of pedagogical efficiency is useless. Or so it might seem.

Now the mistake made by some at this point is to argue that we ought to discard such "value-laden" concepts -- in our case, pedagogical efficiency -- and look for something more "objective," that is, something that will not depend upon a consensus of values in order to be applicable. We can define relatively value-free concepts, but whether they are of educational use is another questions. We have such a neutral concept in "behavioral effectiveness." But is it educationally useful? Consider that any of the following treatments is behaviorally effective: homicide, hamstringing, lobotomy, ... and a host of others one need not even have a fertile imagination to conceive. If we look for some way to rule out undesirable modifications, we end up with something like our concept of pedagogical efficiency (which has the added attraction of a concern with a cost-benefit ratio insinuated by it.)

The bias in the concept of pedagogical efficiency is not a disadvantage, however. Rather, it displays the inevitable concomitant of any educational undertaking: a commitment to some notion of values.

But, you may protest, if two communities cannot agree whether a change in behavior is an enhancement, then they would not be able to settle on whether some method is pedagogically efficient. (In some countries, the maiming of children is done precisely because it makes them more piteous beggars. I cannot even begin to entertain this as an enhancement.)

True. Lacking such agreement on what is an enhancement, two communities will not be able to settle on what is pedagogically efficient. But as least we know where to look for the problem: we will check to see the extent to which there is a consensus of values and understand that its limits limit the scope of our educational undertaking. But only within that scope are problems mere technicalities.

If there is no foundation of common values, we can consider whether to continue to expend our energies in the pursuit of goals not shared by others. We might attempt coercion or deception if consensus is not otherwise attainable. It is probably not too far off the track to understand the Myth of Scientific Education as a clever deception seen necessary for the very fact of our societal pluralism.(12) At very least we may relinquish that self-delusion that has characterized much educational practice this last hundred years: i.e. that a neutral "scientific procedure" is being developed that will unburden us as educators of a commitment to some system of values.

Increasing Teaching Efficiency

The intent, at least, of educational methodology is pedagogical efficiency. The concern to increase teaching efficiency we can understand now to be dangerously ambiguous: what will more efficient teaching bring about? Higher pedagogical efficiency? Or will other interests be enhanced? The analysis in this paper has presented a variety of such interests and knowledge of this variety should frustrate glib attempts to promote teaching efficiency as merely some kind of technical -- read "value neutral" -- efficiency.

How can I be a better teacher? This is an important question; one not to be answered by reference to some catalog of teaching materials. Ask first: better to what end? At what cost? Is this method better than others?

I will sketch a brief assessment procedure for method X.

A. Begin by asking the following questions:

1. Will method X help my students learn the subject better?

2. Is method X ethical?

3. Are the outcomes of method X worthwhile learnings?

4. Is it good for the student to learn them via method X?

5. Is there an advantage to me as a teacher in using method X?

6. Will method X facilitate my functioning with this organization?

C. We ask ourselves of each answer what reason we have to believe our answer is true, independently of our hopes and wishes.

D. We resign ourselves to the disagreements we may have with others as to the answers and justifications for each question. We might examine if our method is "consensually efficient," i.e. it enhances a workable consensus among those persons necessary to its implementation in spite of any deep philosophical questions it leaves unanswered.

This new twist in our procedure clearly seems to indicate that a requirement for good methodological assessment is more good political sense than technological expertise. We can begin to understand the traditional pursuit of a value-free "Science of Education" as deriving from a wish to avoid "politics." (Strange behavior from those who would promote Education for Democracy!)

Public Education is Necessarily Political

This is not a new idea, yet it is quite common to hear educators and other would-be school reformers express the desire to "rid education of politics." Politics should not be seen as a contamination. Rather, we should understand any decision as political if it is based on something other than considerations of efficiency within a framework of a consensus on goals. Thus, most public decisions in a pluralistic democracy will be political. Politics is the art of reconciling disparate ends to common means. It is not an excrescence on education; but, in a democracy, its very soul and substance.

Lest the analysis of this paper end on an overly optimistic homiletic note, it is important to point out some of the likely consequences of the politicization of educational method:

a. the most socially viable methods will tend to low efficiency; this may lead, in times of scarcity, to a rejection of the goals they are instrumental to;

b. "excellence" will tend to be perceived either as an empty slogan, or as an elitist, undemocratic pursuit;

c. teaching will tend to "deprofessionalization" in any politically sensitive arena.

d. expertise will be seen as antipathetic to an increasingly popular concern for "sensitivity to human differences."(13)

Such trends may, in the long run, enable non-democratic elites to gain or to maintain disproportional influence. Education in a democracy, may not -- contra Dewey -- best serve that democracy by being democratic.

(1) P.H. Rossi and H.E. Freeman. Evaluation: a systematic approach. 3rd Edition. (Beverly Hills, CA: Sage Publications, 1985) p.321.

(2) H.L.A. Hart, "The Ascription of Responsibility and Rights" in Anthony Flew, (ed.) Logic and Language. Garden City, NY.: Doubleday Anchor, 1965.

(3) See G. K. Clabaugh and E. G. Rozycki, Understanding Schools. The foundations of education. (New York: Harper Rowe, 1990) pp. 536 - 539 and "Technical Appendix to Chapter 18" or http://www.newfoundations.com/TeLeHTML/TLPallette.html.

(4) For an extended exposition of this controversy, see Sheldon Rothblatt, The Modern University and its Discontents. The fate of Newman's legacies in Britain and America. Cambridge, 1997.

(5) See, for example, James Levin & James F. Nolan, Principles of Classroom Management. A professional decision-making model. (Boston: Allyn and Bacon, 1996)

(6) Jenson mentions that an apparently common criteria of many general intelligence measures is a reliance upon the subjects' ability to make sensory "cross-modal" transfers of information: one learns by listening to what one must later display by writing, for example. Arthur R. Jensen, "How Much Can We Boost IQ. and Scholastic Achievement?" Harvard Educational Review. Winter 1969. Volume 39, pp.1-23.

(7) Jenson in "How Much..." remarks that in a hunting society, IQ as we identify it would be of little value.

(8) The relationship of effectiveness to efficiency is more technical than is advisable for the main presentation. A few considerations indicate that efficiency is the more important concept. Suppose a measure of effectiveness, e, to range from 0 to 100 percent. Suppose effectiveness to be a function of method and constraints, e=f(M,C). Efficiency, E, is directly related to e and inversely related to resource consumption, R, e.g. E=e/R. Since any method takes at least some time, 0<R. But R is never infinite. Thus E = 0 implies e=0. Thus a method which is "not at all efficient" is only one which is not effective. So in the Venn diagrams used herein, 'efficient methods' can substitute for 'effective methods."

(9) A good analysis of needs is found in Leonard Waks, Needs and Needs Assessment: a conceptual framework. (Phila.: Research for Better Schools, Inc. 1979) Monograph. See also, Edward G. Rozycki (2004) "Needs Assessment: a Fraud?" available at http://www.newfoundations.com/EGR/Needs.html

(10) Clearly the point of both Christopher Jencks, Inequality. (New York: Harper, 1973) and James Colemen, et.al. Equality of Educational Opportunity. (Washington, DC: U.S. Govt. Printing Office, 1966

(11) An effective antidote to wishful thinking has long been Christopher J. Hurn The limits and possibilities of schooling. (Boston: Allyn & Bacon, 1978)

(12) Cf. Clarence J. Karier, Shaping the American Educational State (New York: Free Press, 1975) especially chapters 5 through 7.

(13) Given highest priority to a goal of social harmony, the following may be efficient means: ignorance, incompetence, impotence and irrationality. This provides a structural explanation for public school failure. See Edward G. Rozycki, "Values, Rationality and Pluralism". Philosophy of Education 79, 195-204

TO TOP