Improving Schools By Standardized Tests

Front matter

REFERENGE LIBRARY

OF THE

STATE NORMAL SGHOOL

Preface

Tuis book is written for superintendents, teachers, and all other persons interested in the use of educational tests and measurements as a means of improving the work of the schools. It tells how standardized tests and scales were used periodically for two years throughout an entire supervisory district, and how the results of the tests were put to practical use in classifying pupils into grades, for measuring the progress of pupils in their studies as a partial basis for promotion of pupils, for measuring the efficiency of teachers, for measuring the relative efficiency of special methods, and for motivating the work of both teachers and pupils. It also tells of remedial measures, in the way of modification of subject-matter and teaching methods, taken to improve unsatisfactory conditions revealed by the use of the tests.

In its field, the book is unique in at least two respects: (z) it is a narration of actual experience rather than an exposition of theory, and (2) it describes in detail how a comprehensive, periodical testing program was planned and carried out, how the interested codperation of the teachers was secured, and how the results of the tests were used to improve the schools. One whole chapter is devoted to what the teachers and pupils had to say, at the end of two years’ use of the tests, as to how the tests were of help to them in their work.

So far as the author is aware, all the books previously x PREFACE published on the subject of standardized tests and measurements deal almost entirely with the tests themselves; the need for them, descriptions and reproductions of them, and data concerning their derivation and standardization. Very little is said as to how the tests can be put to practical use in the schoolroom, and that little is usually expressed in very general terms. No book heretofore published describes in detail the putting of standardized tests to practical use in a concrete situation and periodically over a period of years.

The above-mentioned books have proved very valuable in making the educational public acquainted with standardized tests and their possibilities. But is it not time that we had something definite in book form as to how the tests are proving of value, or otherwise, in the hands of teachers and administrators? It was with the idea of starting something in this line that the author has set down in this book his own experiences with standardized tests as an aid in school supervision. The book does not claim to settle any of the great problems of modern education. It merely tells how standardized tests were used in an attempt to solve some of the more pressing of these problems as they appeared. In it we tell what we did, why and how we did it, and the results obtained. We have tried to tell it in such a way that any one who so desires may easily follow the general plan with modifications, if necessary, to fit his own particular field of endeavor. It is hoped that the book may prove both interesting and helpful to our fellow workers, and that it may give at least a little added momentum to the great movement toward objective measurement of “classroom products.” If it ac-

PREFACE xi complishes these objects, even in a small measure, the author will feel amply repaid for the time and labor involved in presenting his experiences for publication.

It is with pleasure and gratitude that the author here acknowledges his indebtedness to Dr. B. R. Buckingham for kindly advice and encouragement during the preparation of the work, for many helpful suggestions, for his sympathetic editing of the manuscript, and finally, for the Editor’s Introduction.

S. S. BRooxs WIncuESTER, N.H., July, 1922

Contents

EMAPTERS INTRODUCTION. . 1. -. 3 6 sw’ %

CHAPTER II. THE PracticaL SITUATION. . . . . . Sizing-up the situation — Analyzing the situation.

CHAPTER IIT. GettING TEACHERS TO FEEL THE NEED FOR STANDARDIZED TESTS . . . Pe hats

Chapter I Introduction

EpDucaTION is gradually becoming less of a custom and more of a science. A force of professionally trained educators, imbued with a determination to free the schools from the chains of medieval traditions in subject-matter and method, and with an ambition to apply the principles of common sense to education, is being rapidly built up. The development of modern educational psychology is bringing about changes of management and method that are little short of revolutionary. The ossifying doctrine of formal discipline and a multitude of other hoary traditions have been proved invalid by the scientifically controlled experiments of experts in the psychology of learning. Thus the very foundations have been knocked from under the existing educational system, leaving it, so to speak, very much up in the air.

But if these experts were to be anything more than iconoclasts, they must furnish something better in the way of teaching aims and methods to replace the demolished idols of an educational world whose complacency they had sadly ruffled. Accordingly, they inaugurated in some quarters experimental schools and school systems— laboratories for the compounding of educational doctrines. While still very young, however, the new movement became seriously afflicted with a rash of faddism due to under-trained but over-enthusiastic reformers who mistook their day-dreams for facts of proved worth. Having passed through this period, however, the movement began to steady itself. Earnest seekers after the truth early came to realize that to foist half-baked schemes on an ultra-conservative public would in the end merely breed distrust and suspicion of all new ideas in education regardless of their value. Reformers grew more cautious of presenting their ideas until they had proved their worth. Local school men and women also became more cautious of introducing new methods until those methods had been successfully tried out under fairly representative conditions. Thus something like a healthy balance was struck between faddism and conservatism.

Among the more recent ideas there are some that seem pregnant with tremendous possibilities for the improvement of educational procedure. ‘The most far-reaching in its beneficial possibilities seems to me to be the idea of measuring the results of teaching and the intelligence of school children by means of standardized tests. Of large. importance also are: “The Socialized Recitation,” “Supervised Study,” “Teaching Children How to Study,” ‘The Project-Problem Method in Teaching,” and the change of emphasis from oral to silent reading. All these things are rich with promise of greater efficiency in education. But although all the above-mentioned ideas have proved successful, when carried out by trained men and women full of enthusiasm and thoroughly understanding the psychological principles upon which the ideas were based, nevertheless, when the carrying-out of these same ideas has been attempted by persons lacking in sympathy and full understanding of the principles, aims, and methods involved, they have often been far from successful.

INTRODUCTION 3

Superintendents and teachers who condemn these ideas, after proving to their own satisfaction that they are worthless, remind me of the first farmer in the town of my boyhood to purchase a mowing machine. For a great many years this man had mowed his many rough and rather rocky acres by hand with the help of a large crew of hired men. But one spring the mowing-machine agent appeared in our town and as a result, by the time hay was ready to be cut, the farmer found himself possessed of a brand-new mowing machine, resplendent in green, yellow, and red. The next morning he started out to mow the “ back field.” Truly his trials that day were many. And when the machine, unable to stand any more of his inexpert handling, broke down late in the afternoon, he had succeeded in getting over only about as much ground as one good man with scythe and snath could have mowed in the same length of time. And what an untidy-looking job it was! The next day the farmer started his old crew to mowing by hand. No more of that mowing machine for him. He could do more and better work in a day alone than could a man with a pair of horses and a mowing machine. The mowing machine was a failure. He knew because he had tried it. But did this farmer’s experience with his new mowing machine prove the failure of the machine or the failure of the man? ‘The mowing machine, we know, will do good and rapid work in astonishingly rough and rocky ground when guided by a practiced hand.

And so it is with new methods and ideas in education. The socialized recitation can easily result in chaos in the hands of weak teachers. Supervised study can become the sepulcher of all energy and initiative on the part of pupils if improperly conducted. Teaching children how to study becomes a farce when attempted by teachers who do not themselves know how to study efficiently. And how many of them do know how to study to the best advantage? As for silent-reading — well, some of its ardent exponents would get the surprise of their lives if they sent out questionnaires to all the teachers of so-called silent-reading asking them to explain their conception of silent-reading drill.

For example, I visited a village school not long ago, where, the superintendent told me, one of his best teachers was working out some of the “ new ideas.”’ I found the teacher industriously correcting papers at her desk. ‘The room was very quiet and orderly, and every child was busy.

“Do you teach silent-reading here?” I asked, after introducing myself. :

“Oh, yes! That is what we are doing now. We have silent-reading drill for a whole hour every afternoon.”

_“ And is that all the silent-reading drill they get?”

She looked up in surprise. ‘‘ Why, yes, that seems to be as much time as we can spare without neglecting the other work. The pupils enjoy it very much,” she added.

And why should n’t they enjoy it? The school was well supplied with children’s story-books and for a whole hour every afternoon the pupils were permitted to forget the serious affairs of life and to dawdle over story-books without aim or purpose. Was this efficient silent-reading drill, or was it largely a waste of time and money?

Now the moral of all this is that neither a machine nor an educational method should be condemned as worthless because it fails in unpracticed hands. The successful working-out of any of the things mentioned above (especially in the present state of our knowledge about them) demands energy, initiative, industry, enthusiasm, and a fairly complete understanding of the psychological principles involved together with ability to apply them. The form of a method without its spirit will not ensure success

INTRODUCTION 5

Before introducing a new mode of procedure into a school or school system, superintendents and teachers should be sure that they know what they are going to try to do, why they are going to try to do it, and what others have done along the same line. In addition they should have a definite plan as to how they expect to doit. This plan may have to be changed or modified several times before results are satisfactory, but nevertheless, some definite plan should be provided at the start.

For information as to what others have done one must have recourse to educational books and _ periodicals. Much has been written, in one form or another, concerning supervised study, teaching children how to study, silent-reading, and standardized tests. Most of these writings, however, deal with generalities. Now, either through lack of inherent ability or through lack of proper training, it is undoubtedly hard for most people to apply general principles to specific cases; to take a suggested plan, think out the details, and put the plan into successful operation. This is the reason for such a large crop of failures when a superintendent calls his teachers together, tells them what he wants done, explains the general principles of the new plan, and leaves them to work out the details and produce results.

The research men are giving us some splendid books as a result of their years of study and painstaking experiment. Their statements of procedure are, for the most part, sound in theory; and many of them have been proved by practice — especially when conditions were favorable. But these general principles, when applied: under widely different and often adverse conditions, do not always work well. There are, however, some school people who are not easily discouraged, who take the trouble to make a careful study of the causes of failure, and who modify either the plan or the local conditions or both; and it is they who put the results of research into successful operation.

Would it not, therefore, be well if we had more printed records direct from the actual workers in the field — from the men and women at the head of local schools and school systems and from their teachers? Even if these records were not of high technical or literary merit, they would tell just what teachers and superintendents have tried to do, and why and how they have tried to do it, together with the results achieved. ‘The story might not always be one of shining success, but could it not be valuable to fellow workers for all that? If we are truly wise, we may often learn success from others’ failures. Most of us profit by reading about the experiences of others in our own line of work, even though we are sometimes foolish enough to think we have nothing more to learn.

Hence the excuse for this book. It tries to tell in a simple way how standardized tests and scales were used to improve the schools in a newly organized rural-school district in New Hampshire under conditions that were, to say the least, discouraging. In a way it is story. It is the story of how a corps of faithful, hard-working, but mostly untrained teachers, with the aid of an inexperienced superintendent, put standardized tests and measurements to practical use throughout a school system to the considerable advantage of all concerned; of how also, in connection with the use of the tests, they solved, with at least some satisfaction to themselves, the problems of efficient silent-reading drill, supervised study, and teaching children how to study, in one-teacher rural schools. It is not a story of complete and unqualified success; but I have endeavored to make it a faithful, readable, and understandable account of what was done, of why and how it was done, and of the results achieved.

INTRODUCTION 7

The book is addressed to teachers and superintendents, both urban and rural. The principles of procedure set forth can be applied in a single school or a whole school district, in large schools or small, in city or country. The methods used have been tried out under about as unfavorable conditions as could be found anywhere. Although the actual experiments have been carried on in a purely rural district, I can conceive of no reason why the same principles cannot be applied with even greater facility, and the methods used to even greater advantage in town and city schools. Many of the obstacles that face the rural superintendent in attempting such a testing program as is here described are largely absent in the city. Bettertrained teachers are available; the teaching force is more stable; and it is much easier to get the teachers together for frequent meetings. ies

-

Chapter II The Practical Situation’

A SUPERVISORY district containing twenty-six rural, ungraded one-room schools with teachers for the most part inexperienced or untrained, a majority of whom had never worked under a superintendent before— this was the opportunity that fell to my lot when I became a New Hampshire superintendent under the new law effective September 1, 1919. It was my first experience as a superintendent. Truly the district was “ virgin soil” as a member of the State Department remarked to me when I took the position.

SIZING-UP THE SITUATION

THE first two or three weeks were spent in traveling about the district getting acquainted with the teachers and sizing-up the situation. Most of the schools had already begun, the teachers having been hired by the local school boards as in the past. I found that three of the teachers were normal-school graduates, and that two had attended one summer school; but the others had no professional training whatever. Several had not attended school beyond the eighth grade, but had taught from ten to twenty-eight years in the same school. Three were inexperienced high-school graduates in their teens. Fifty per cent of all the teachers had never seen a professional book or magazine and did not know where to obtain one. All this, of course, had resulted in an ingrowing provincialism which could not but have a disastrously narrowing effect on their teaching. They were imitators of imitators.

THE PRACTICAL SITUATION © 9

Their methods were in imitation of the teachers who had taught them and who, in their turn, had imitated their own teachers. The results were the use of methods and texts so archaic as to be amusing if they had not been at the same time such a sad commentary on our boasted educational system.

Most of the textbooks were sadly out of date. erie tics, geographies, grammars, and even histories published in the eighties were in daily use. One of the local committees insisted that they must be good books or they would not have worn so long. Of course the books were all based on the defunct doctrine of formal discipline and showed little of the psychological methods of presenting material which are the basis of modern educational procedure. The idea seemed to be that an arithmetic is an arithmetic, and that a geography is a geography, one book being as good as another so long as nothing pertaining to the subject is omitted and the leaves are all present.

I have emphasized the teaching and textbook situations because I consider them the most important factors. I shall mention a few other points briefly. As for the schoolhouses, it is sufficient to say that most of them were typical New England rural-school buildings of ancient vintage, modeled variously (according to the ideas of their instigators), upon churches, town halls, barns, and in some cases it would almost seem, upon pigpens. ‘They were small, dirty, poorly heated, lighted. and ventilated, and in short generally unsanitary and ill-suited to their purpose. As to the organization of the schools, as I have already intimated, no attempt had ever been made to grade them. Another important factor in the situation was the attitude of the communities toward expert supervision. Three of the four towns in the district had always opposed such supervision as long as it was optional and were not inclined to submit gracefully when, according to the provisions of the new law, it became compulsory. Such were the general conditions as revealed by my preliminary survey.

However, there were three bright spots in the general darkness of the situation. First, the splendid new State school law which made the superintendent a State official with a pretty free hand and which provided liberal State aid for rural schools, where it might be necessary, to keep them up to the required minimum standards. Secondly, intelligent school boards willing to codperate. And last, my own professional equipment which included a pretty thorough training in educational measurements and an earlier experience in the teaching of rural schools. Accordingly, I tackled the proposition determined to show what scientific method in education, as I understood it, could do for rural schools if it had the chance.

ANALYZING THE SITUATION

TuE following week was spent in analyzing the conditions found to exist and in deciding upon the most efficient way to bring order out of chaos. Asa result of this analysis a number of definite problems seemed to stand out clearly demanding early attention. From among these problems I shall select, for present discussion, only those, in the solution of which, as it seemed to me, standardized tests and scales could be used to good advantage.

Problem 1. To grade the schools fairly and accurately. Getting the schools graded in order to start the pupils right for their year’s work was of course the most urgent problem. It would have been a comparatively simple matter to grade them arbitrarily on the basis of the teachers’ judgments. But there are serious objections to this method even as a beginning, especially when the teacher

THE PRACTICAL SITUATION I is new to the school. If, for example, children are placed in grades lower than their parents think they ought to be, there are bound to be strenuous objections from some of these parents, and the only evidence one has to offer is that the teacher thought the children ought to be placed in those grades. Did you never hear this line of talk from an irate parent? ‘“‘ My John is just as smart as Mrs. Smith’s Henry and he ought to be in the same class or above. It’s a plain case of showing partiality. I just won’t stand it!”

It is not by any means easy in such cases to prove to the satisfaction of the parent that the teacher had a sound basis for her judgment. And the fact is that she did not have a sound basis. How did she know just what knowledge or ability a child should exhibit in order to belong ina certain grade? What standards of achievement did she have for the different grades besides her own arbitrary judgment?

Giving ordinary tests for grading purposes is also inaccurate and unfair, because such tests are devised by the teacher or superintendent and hence represent merely the judgment of one of them as to what he or she thinks a child ought to know in order to be placed in a certain grade.

Worse than all else, the children themselves are apt to become discouraged, not understanding wherein they have failed, and being at a loss as to just what they are expected to achieve. Even the superintendent and teachers, if they are conscientious, are not satisfied because they cannot feel sure that injustice has not been done. Retarding a child without good reason, thus robbing him of a year of his time, is a serious matter, at least for the child.

What method of grading, then, could be found which would be fair to all concerned? After casting about for some time for a solution to this problem, it suddenly occurred to me that standardized tests would help me out of my difficulty. Did they not offer definite standards of achievement for each grade? And those standards were not based on an unattainable one hundred per cent nor upon the opinions of dogmatic educators as to what a child ought to know at the end of a certain grade. They were experimentally derived and based upon the amount of work that normal children are actually doing in the different grades throughout the whole country and not upon the amount of work somebody thinks they ought to do.

By giving these tests in all my schools I thought I could determine just where each child belonged on the educational ladder. The process would be roughly comparable to measuring a large number of sticks of various lengths by means of a tape measure sorting them into eight piles, each pile containing sticks of about the same length. Then, when angry parents wanted to know why their children were graded so low, I could show them. The children themselves could be made to see wherein they had fallen below standard and what they would have to do to come up to the standard. Both superintendent and teachers would feel that they were on solid ground. Surely I needed standardized tests the first thing.

Having reached this conclusion, I immediately ordered all the tests and scales I knew of that were sufficiently well standardized for my purpose. They covered the following subjects: arithmetic, silent and oral reading, spelling, writing, geography, history, and English language.

Problem 2. How to measure the progress of pupils. Having settled upon how to do the preliminary grading,

1 For those who do not know standardized tests and want to get acquainted, J recommend for a beginning the purchase of Measuring the Results of Teaching by Monroe, and Educational Tests and Measurements by Monroe, De Voss, and Kelly.

Both books describe the best tests and their uses and the former tells where to obtain them. These books are published by Houghton Mifflin Company.

THE PRACTICAL SITUATION 13 the next problem was to find a just method of measuring the progress of pupils for promotion purposes. The same objections to teachers’ tests hold good here as were. noted in discussing the previous problem.

This problem involved, obviously, the finding of a satisfactory system of marking. It has been proved beyond doubt that ordinary teachers’ marks are unfair, inaccurate, and generally unsatisfactory as a means of measuring progress or as a basis for promotion.

In measuring the progress of pupils, standardized tests must surely find their widest field of usefulness. Here the process can be compared to standing a ten-foot measuring rod beside a young tree when it is a foot high and noting its growth in height. When it has added another twelve inches it belongs in the two-foot class. Twelve inches more and it is promoted to the three-foot class, and so on. It is entirely possible to devise a satisfactory system of marking, with the aid of such tests, as will be explained later on.

Some of the teachers thought that the new State program of studies for elementary schools would be a sufficient guide for promotion purposes. This program is thoroughly practical and up-to-date. But it offers no definite, objective standards of achievement from grade to grade. Therefore it is not a satisfactory tool with which to measure the progress of pupils and to determine when they are ready for promotion. It is true that a printed program of studies specifies the subject-matter to be covered by each grade, but it is left entirely to the judgment of the teacher as to when that subject-matter is covered satisfactorily. The ground may be gone over more or less thoroughly and with very unequal results according to the methods and arbitrary standards of the individual teacher. Hence, it is obvious that programs of study cannot take the place of standardized tests that are even fairly objective.

Problem 3. How to measure the ability of teachers. One of the greatest needs of superintendents is a method of measuring the ability of teachers that will be accurate and fair to all concerned —a method that will not leave the superintendent open to the charge of favoritism or poor judgment — a method that will satisfy a teacher who has been rated something less than “ excellent” that her shortcomings are real and not hallucinations in the mind of the superintendent due to his personal dislikes and prejudices.

What is wanted is concrete evidence of a teacher’s ability or lack of ability that will allay carping criticism from whatever source. It is decidedly neither accurate nor fair to estimate a teacher’s ability solely by observation made by the superintendent during his visits to the classroom. It is not fair because (a) such observations do not furnish a sound basis for judgment; (0) the superintendent’s opinions are quite apt to be colored by personal prejudices toward an individual teacher or her methods; (c) classes often show at their worst in the presence of visitors; (d) even the teacher may fail to do herself justice under the critical eye of the superintendent. It is inaccurate for all the reasons noted above and because (a) some teachers do excellent work when the superintendent is present and shirk all the rest of the time, and (0) if such teachers do their own testing, even the results may be made falsely to appear satisfactory.

Provided a teacher is of good moral character with high ideals and a fairly pleasant personality, her further desirability as a teacher is measured by the results she gets as determined by the progress of the pupils when such progress is objectively measured. After all, it is results

THE PRACTICAL SITUATION 15 we are after, primarily. Hence, the standardized tests measure objectively both the progress of the pupils and the ability of the teacher at one and the same time.

Problem 4. To find a practical method of supervising study and of teaching how to study. This may seem like a large contract for the one-teacher rural school with its crowded curriculum, but I shall try to show that it can be successfully worked out by using proper methods of study supplemented by judicious use of the standardized educational tests and measurements.

This chapter merely presents some of the problems confronting me in my new work together with some plans for their solution and reasons therefor. The following chapters will describe in detail the working-out of these plans and the results achieved.

Chapter III Getting Teachers To Feel The Need For Standardized Tests

Havinc concluded that standardized tests would greatly help me under the conditions set forth in the preceding chapter, I next attacked the problem of securing the wholehearted codperation of my teachers in their use. The psychology of interest teaches us that a person’s best efforts are called forth only when he feels a real need. So I set myself to arouse in my teachers an enthusiastic interest by bringing them to feel a real need for definite standards of accomplishment in school work. In order to make them feel this need deeply, something more than merely telling them of the tests and their uses was necessary. They must be made to realize the inadequacy of ordinary methods of measuring the results of teaching.