Introduction to the use of standard tests; a brief manual in the use of tests of both ability and achievement in the school subjects

Introduction To

THE USE OF STANDARD TESTS

A BRIEF MANUAL IN

THE USE OF TESTS OF BOTH

ABILITY AND ACHIEVEMENT

IN THE SCHOOL SUBJECTS

* * * By SIDNEY L. PRESSEY, Ph.D.

Assistant Professor of Psychology The Ohio Stale University

AND

Instructor in Psychology

The Ohio State

University

Yonkers-on-Hudson, New York WORLD BOOK COMPANY

WORLD BOOK COMPANY

THE HOUSE OF APPLIED KNOWLEDGE

Established, 1905, by Caspar W. Hodgson

yonkers-on-hudson, new york 2126 Prairie Avenue, Chicago

The aim of the World Book Company is to publish books that apply the world's knowledge to the world's needs. The need for knowledge of the ways and means of using and profiting by standard tests is the one that is felt more than any other at the present time by those engaged in educating the children of the country and the world. A great fund of information as to the use of standard tests is possessed by those who have specialized in this subject. The authors of this book are such specialists, and they seek to apply this knowledge to the needs of the busy teacher, principal, and superintendent, who are increasingly realizing the importance and the necessity of mental and educational tests.

Copyright, 1922, by World Book Company

Copyright in Great Britain

All rights reserved

PRINTED IN V. 8. A.

Contents

CHAPTEB PAGK

Introductory 1

Part One — How To Use Tests

I. What Is a Test? 7

1. Choice of subject matter of standard tests 8

2. Procedure in giving, taking, and scoring 11

3. Saving of time and labor 17

II. School Problems and Tests 20

1. Tests and the problems of the teacher ... 22

(a) Problems involving the measurement

of general mental ability 22

(b) Previous preparation in the several school subjects 24

(c) Diagnosis of special weaknesses . . 24

(d) Remedial instruction 26

2. Tests and the Problems of the Supervisor 27

(a) Progress of pupils in school subjects 27 (6) Comparison between classes and

schools 27

(c) Determination of the causes of differences between classes 28

(d) The adequacy of teaching methods . 28

3. Tests and the Problems of the Superintendent 30

(a) The need for a measure of efficiency 30

(b) Determination of accuracy of grade placement 31

(c) Tests and educational reorganization 32

(d) The need for convincing facts ... 33

(e) Comparison of entire systems ... 33

III. The Common Sense of Statistics .... 36 1. Simple calculations made by arranging

papers in order 37

iii iv CONTENTS

CHAPTBB ' PAGE

2. Tabulating, and interpretation of tables . .41

3. Expression of relationships 48

4. Application of these statistical methods to practical problems 53

IV. Use and Misuse of Tests 60

1. Necessity for standard procedure in using tests 60

2. Special factors influencing test results . . 64

3. Problems of interpretation . . 69

Part Two — Tests In The School Subjects

V. Tests in Arithmetic '. ■■ 77

1. Procedure in considering tests ...... 78

2. General tests in arithmetic 80

3. Diagnostic tests in the fundamental operations of arithmetic 82

4. Practice materials in the fundamental operations of arithmetic 85

5. Tests in problem solving 88

VI. Tests in the Content Subjects — Geography and History 91

1. Tests in geography 92

2. Tests in history 94

VII. Measurement of Ability in Written

English 97

1. Measures of general merit in English

composition 98

2a. Diagnostic tests: Ability in spelling . . . 102 2b. Diagnostic tests: Knowledge of punctuation and grammar 104

3. Other test materials having to do with written English . . . 106

CONTENTS v

CHAPTEB PAGE

VIII. Tests in Reading 108

1. Tests in oral reading 109

2. General tests in " silent reading ability " or thought getting Ill

3. Diagnostic tests in " silent reading " . . .115

IX. Measurement of Handwriting 121

1. General measurements of skill in handwriting 122

2. Diagnostic measures in handwriting . . .125

3. Systematic practice exercises in handwriting 128

X. Tests in the High School Subjects . . . 131

1. Tests in English 132

2. Tests in algebra 133

3. Tests in geometry 134

4. Tests in Latin 135

5. Tests in the modern languages 138

Part Three — Tests Of Mental Ability

•XI. The Measurement of General Mental

Ability 145

1. The general nature of the tests 145

2. Scales for individual examination .... 149

3. Scales for group examination 154

4. Tests of special abilities 162

XII. Use of Tests of General Mental

Ability 165

1. Limitations of such tests 165

2. Use of individual tests in the schools . .169

3. Use of group tests in the schools 173 vi CONTENTS

Part Four — Important General Principles Regarding Tests

CHAPTEB . PAGE

XIII. How Tests Are Made 181

1. Definition of the problem :...... 182

2. Selection of the test form 185

3. Construction of items, preliminary trial, and selection of items for final form .... 190

4. Final trial, standardization, and validation 193

5. Special problems involved in combining several tests into a single examination . . 195

XIV. The Testing Program 198

1. Selection of a project 198

2. Selection of tests 200

3. Organization of test work 205

4. Planning the testing program 208

XV. Making the Testing Program Worth

While 214

1. Enlisting the cooperation of the teachers . 214

2. Relating the test results to the teaching . 217

3. Verifying the usefulness of tests 222

4. The long-time testing program 225

Appendixes:

A. Finding the Median for Large Distributions . . . 227

B. Tests and the Diagnosis of Feeble-mindedness . 231

C. Suggestions for Further Study 235

D. References for Further Reading 239

Index 257

Introduction To The Use Of Standard Tests

INTRODUCTORY

NO longer ago than 1908 appeared the first systematically organized form of the Binet Scale. In 1910 was published the first of the modern scales for the measurement of classroom products — Thorndike's Scale for Measuring the Quality of Handwriting. In 1915 Otis devised the first group scale for measuring general intelligence, embodying the principles later used in the army group tests. These dates may be taken as marking the real beginnings of the present efforts to measure mental ability and school attainment, though of course there had been scattered work in both lines before these years. 1

The measurement movement, therefore, is comparatively new; in fact, the most important developments have come during the past few years. Particularly did the extensive psychological work done in the army during 1917 and 1918 give a new impetus to the movement, and establish new methods in testing. The testing of over one million draftees, and the use of test results in placing these men where they would be of the greatest value to the service, demonstrated the usefulness of these methods in a striking and dramatic way. As a result of the army work, and of continued

1 Thus Rice's report on spelling in 1897 may be considered the first effort, in this country, to investigate empirically the product of teaching, and Cattell's paper in 1890, dealing with his tests of Columbia freshmen, the beginning of measurements of ability.

trial of tests in the schools, it may be said that the usefulness of the test method is now an established fact. And tests are being used to an extraordinary extent, both in the public schools and in the selection of employees by business houses. A large number of excellent tests are now available; methods are gradually becoming standardized.

Now that tests are being used very extensively, by teachers and others not especially trained in their use, there is needed a simple and direct manual, covering tests of both ability and achievement, to make clear to such workers the fundamental facts with regard to tests, the handling of test results, and the significance to be attached to the results of testing. Particularly there is need of a systematic treatment of the ways in which test results may be used in dealing with school problems. The present brief text aims to fill, in some measure, the need for such a manual.

The effort has been first of all (1) to give a clear discussion of the nature of tests, the problems that may be dealt with profitably by means of tests, simple methods for the handling of test results, and common mistakes to be avoided. Every effort has been made to keep the treatment non-technical, and to make it of the most practical character. In the second part of the manual (2) certain of the best and most representative tests in the various school subjects are presented. This presentation has been made brief, since, with the present activity in this field, improved instruments are constantly appearing and present scales and tests will become out of date. Therefore attention has been concentrated rather upon the fundamental problems of

INTRODUCTORY 3

measurement in the various fields, and present tests have been used as illustrative material. In the third part (3) tests of ability, particularly tests of general intelligence, have been discussed in the same way, with special emphasis on the practical uses of test results. Finally (4) certain general principles with regard to the way in which test work should be organized, and related to practical problems, have been emphasized. The manual is thus intended as an introductory handbook in the use of tests. The effort has been, throughout, to suit the book to the needs of the busy teacher, principal, or superintendent.

CHAPTER ONE What Is a Test?

BEFORE the appearance of the standard tests and scales with which this book is to deal, "test" was synonymous with "examination" or "written review." A test consisted of a number of questions selected by the teacher, and covering what she thought to be the important points of the subject studied. But with the introduction of the standard scales the word "test" has taken on a new meaning — a meaning which, superficially, appears very thoroughly to differentiate it from the ordinary school examination. Many teachers and administrators are still skeptical of the value of the new "tests" advocated by educationalists and psychologists. These practical schoolmen are skeptical not only of the practical value of the results obtained by use of these tests, but of the essential soundness of the present test movement. There is a feeling that testing is a time-consuming, artificial, and one-sided way of obtaining information which the teacher may acquire much better by her own observation of her pupils and their work. These circumstances make it necessary that the exact nature of the tests and the advantages of the test method should be thoroughly understood. It will be found the fundamental contention of this book that tests, as the term is now used, are not merely a very special method to be used only on special occasions. Instead, the new tests are the result of an effort to avoid certain limitations and failings of the usual written examinations. And it will be argued that, far from being a special method to be used only rarely, tests should be part of the equipment of every school, because tests will do much that has heretofore been done not at all or not so well by the usual examinations given by teachers or principal.

Three distinctive features of the "test" as compared with the usual school examination, will be pointed out.

1. CHOICE OF SUBJECT MATTER OF STANDARD TESTS

A good test covers only the really important points of a subject. A teacher intends to cover only important points, but no one person can have an unbiased opinion as to what is important. The maker of a test consults as many textbooks as he can find, talks with as many teachers of the subject as possible, corresponds with advanced students and experts in the subject under consideration, and so comes to a conclusion as to what points, in the total amount of subject matter, are of greatest importance. He then discards those questions that seem of relatively little significance and keeps only those which deal with basic facts. The subject matter of the tests is, then, far more carefully considered than are the questions of a school examination. The writers themselves have been guilty of making up the questions for an ordinary classroom examination while riding in the street car on the way to class; and they know some teachers who announce there will be a written lesson and then turn to the blackboard and write down such questions as occur to them while writing. It is evident that even the careful reflection of one person — and such reflection is not infrequently missing — cannot produce questions of so consistently important a nature as can the thoughtful consideration of the problem by many people.

For instance, the Hahn-Lackey Geography Scale consists of a series of questions very carefully chosen. First, questions common to six modern textbooks were selected; only the essentials of the subject matter were thus included. This list of questions was also checked by study of the curricula worked out by special students of geography teaching in the public schools. Then the wording of these questions was very carefully examined to eliminate expressions that were over-technical. And the exercises were revised so that they contained about an equal number of "memory" and "thought" questions. It should be obvious that questions thus carefully selected are much more valuable than questions which might be chosen by a single teacher, no matter how careful she might be in making up her examination.

The best tests are based on very careful research as to the fundamental objectives in the subject concerned, and the material is selected with reference to its importance for these objectives. Thus, before constructing the Ayres Spelling Scale its author determined, by very careful and extended investigation, the words actually used most commonly by children and adults, in written work. The scale was then made to include only the one thousand words used most frequently. Clearly these are the one thousand words which it is most necessary that children should learn to spell.

Moreover, not only is the importance of the questions in a standard test determined, but also their relative difficulty. This is evidently of decided importance. The questions of a test should be neither too easy for the most capable children in the class nor too hard for the poorest children. Preferably the questions should be arranged in order from the easiest to the hardest.

And the increases in difficulty should be regular. That is, Question 6 in the test should be as much harder than Question 5 as Question 5 is harder than Question 4. Actual experimentation has shown that it is practically impossible for a teacher to estimate the difficulty of questions; a question which she thinks easy, and to which she assigns little credit, may really be hard for the pupils. 1 The difficulty of each question in a standard test has been carefully determined by trial with a large number of school children. The test thus contains materials suitable for all degrees of ability in the grades to which it is applicable; there are some questions easy enough for the poorest pupils and some questions hard enough to try the mettle of even the best students. And the regular increases in difficulty make the test into a "scale" in which the units are, in a very important way, equal. And we can say that a child who scores 5 on the scale is as much better than the child who scores 4 as he is poorer than the child who scores 6; the child with the score of 5 does work as much more difficult than 4 as it is easier than 6.

Thus, Monroe's Silent Reading Tests begin with items which are very easy and progress by comparatively even steps to items of decided hardness, for the children to whom the test is supposed to be given. And a child is credited more for doing a hard item than for doing an easy one.

A "test" is thus to be distinguished from the usual school examination first of all by the extreme care used in the selection of materials for the test. The questions

1 Comin, Robert, "Teachers' Estimates of the Ability of Pupils." School and Society, Vol. 3, page 67, 1916.

for a test are selected only after very careful study of textbooks and consultation with authorities, in order to get a consensus of teaching practice and of expert opinion regarding the subject matter to be covered. Where possible, selection of material is based upon searching analysis and extended investigation regarding ultimate objectives in the teaching of the subject in question. And the difficulty of the problems to be used is very carefully considered, so that either a regular progression in difficulty, or a series of problems of equal difficulty, may be secured.

2. PROCEDURE IN GIVING, TAKING, AND SCORING

THE "TEST"

The way in which the test should be given is specified in detail. Particularly in the way they give examinations, teachers differ strikingly. If the reader wishes to experiment with this point, let him write out a series of questions in some subject, and then ask several teachers of the same grade how much time they would give their pupils for answering these questions, and what their procedure would be. He will find that some teachers will allow twice as much time as other teachers will consider sufficient. He will also find that some teachers will allow the children to ask questions if they do not understand what they are to write, whereas other teachers will allow no comments. If he were to watch the various teachers while they gave the examination, he would find that their methods were very different, some reading the directions only once and perhaps hurriedly, while others went over them in detail two or three times. It is evident that the results from the same set of questions from two different rooms cannot be compared if the children in one room are given more time than those in another, or if the teacher in one room gives directions for answering the questions that differ in any essential way from those given by the teacher of the other room. Every one who has taught knows that classes will do vastly different work if told to "work rapidly" from what they will do if told to be "as accurate as possible." And any encouragement given by the teacher, even if only a request that they do their best, will influence the work done by the children. Consequently, it is obviously impossible to compare the papers from children in different rooms, schools, or cities unless the method of giving the tests — the procedure — is identical for all classes tested. Many a teacher wishes to know whether or not her class ranks as well in a subject — say arithmetic — as the class taught by another teacher in another building. But a comparison of the two classes is not justified unless both the questions asked and the procedure used in the giving are identical. The standard tests avoid these difficulties. The directions for giving a standard test are to be read verbatim, as prescribed by the author of the tests. With the best tests, the directions are printed on the test blank, so that failure on the part of the teacher to read the directions distinctly will not influence the results. The conditions for testing are thus rendered perfectly constant — or identical — from class to class, no matter where or by whom the test is given.

Thus the directions for the Courtis Arithmetic Test, Series B, are printed just above the examples, and consist of the following explanations: "You will be given eight minutes in which to find the answers to as many of these addition examples as possible. Write the answers on this paper directly under the examples. You are not expected to be able to do them all. You will be marked for both speed and accuracy, but it is more important to have your answers right than to try a great many examples." In giving the test the teacher has the pupils read these directions aloud with her. And, since the directions are on the same page with the test itself, the children can always glance back at the instructions if they find themselves in doubt as to what to do. So each child has the same opportunity to understand what is to be done and works with exactly the same directions — directions which are presented in exactly the same manner to every other child who is given these tests, whether in the same room or school or not. And the time allowed each class is always the same.

There is also careful control of all factors which might, after the directions were given, affect a child's work on the tests. Thus a history test should require little if any writing on the part of the pupil. The reason for this is simple. If there is much writing, the child who writes rapidly will get more questions done and will, therefore, have a greater chance to get a larger number of questions right than another child who knows just as much about history, but writes so slowly that he has time to answer but a few questions. In any case, the ability to express oneself quickly and adequately in written English is an ability which varies enormously from child to child; and, if a teacher is trying to measure a child's knowledge of historical facts, she does not wish to have the results obscured by such accidental factors as speed of writing. Every teacher has some children who never do themselves credit on an examination because they cannot write down their ideas readily. In order to avoid such difficulties the best "tests" require little or no writing from the child. Instead, the child underlines words, crosses out words, or does something of that sort which will not give any considerable advantage to the child who writes rapidly or the child who expresses himself as easily on paper as in a recitation.

Thus a vocabulary test included in the Haggerty Examination "Sigma 3" includes such lists as:

calm (quiet, sleepy, night, restful)

cupola (church, high, schoolhouse, rounded dome) and the children are told to "draw a line under the best definition for each word." If the children were simply given the words "calm" and "cupola," and were asked to write a definition, the child who wrote the most rapidly would get the most answers done within the time limit, though he might not know as many words as a slower writer. Further, if the child who knew the meanings of the words most exactly defined them most fully, he would define fewer words within the time limit than the child who simply gave a rough definition for each word and then rushed on to the next. By offering a choice of answers, among which the child is to indicate the correct one by underlining it, all these various difficulties are largely done away with.

The scoring of a good "test" is quite as unmistakable and clearly defined as the procedure in giving. In the newest tests answers are not written; certain marks — underlining, crosses, or checks — serve to indicate the correct reply. At most, only a word or two, or a number, serves for an answer. There is thus little opportunity for a mistake in grading, or for doubtful responses on the part of the child. Either the correct word or number or mark appears where it should, or it does not; the answers are always unmistakably right or wrong. The great exactness and objectivity thus possible in grading tests should be compared with the very loose methods usually employed in marking ordinary examination papers. Suppose a teacher of history, in a junior high school, gives an ordinary examination to her class. The mark a given boy receives is almost certain to be dependent not entirely upon the amount of history he knows; the neatness of his paper, the legibility of his handwriting, his facility in written expression, are almost certain to influence the teacher's mark to some extent. Moreover (a very important point indeed), the mark assigned by the teacher is also dependent upon her standards as to what should be required of a class, her judgment as to the comparative importance of various points, and her conception as to the worth of various types of answer. In spite of the best of intentions the mark may be influenced somewhat by her mood at the time, and perhaps by her attitude toward the boy in question. In contrast the score which a boy obtains on a modern "test" is always fair and impartial, and uninfluenced by the special factors just mentioned; the score is thoroughly objective.

Any teacher who doubts what has just been said about the unreliability of marks should try two or three simple experiments. She should, for instance, give an examination and mark the papers, keeping the marks on a separate sheet, — and then put these papers away in a safe place. After the lapse of a month or so she should grade these papers again. Many of the second marks will be found surprisingly different from the marks first assigned. And, if a second teacher is asked to grade these same papers, still more surprising discrepancies will appear. The matter has, in fact, been very carefully investigated. 1 And it has been proved that a group of mathematics teachers may vary all the way from 28 to 92 in the marks they assign to the same final examination paper in geometry. It should be obvious that no such variations are possible in scoring a good test. Thus, in grading the vocabulary test mentioned above there is almost never any question as to whether or not a given reply is correct; either the correct word is underlined or it is not.

To summarize: the second great advantage which a "test" has over a school examination is that the procedure in giving, the nature of the child's answers in taking, and the process of scoring are very carefully worked out and can be kept constant from one class to another. As a result, it is possible to compare the work of one class with the work of another, with a definiteness which was undreamed of a few years ago. Norms, or "average scores," based on the work of several thousand children of a given age or grade are available, for the best tests. So a teacher may compare the standing of her pupils not only with the records made by other classes in her own school or city but with the general average or "standard" obtained from schools all over the country. The teacher of a little country school can find out whether or not her four sixth-grade pupils can do, in arithmetic, or spelling, or history, what has been decided upon as "average" for pupils of that grade in school systems generally. From the point of view of comparability of results the standard test is unique.

1 Starch, Daniel, Educational Psychology, pages 426-438.

3. SAVING OF TIME AND LABOR

The statement that the test method saves time and labor will be immediately challenged by many teachers who may have slaved for hours over the scoring and interpretation of some awkwardly constructed test. Certainly, not all the tests now on the market are easily given and scored. However, it may be said shortly that a good test should save time and labor.

Consider, for instance, the questions in American history which appear below:

The people who settled Plymouth were: Dutch English French German. The Quakers settled in: Quebec Maryland Vermont Pennsylvania. The Spaniards explored America chiefly to obtain: furs gold trade land.

This eighth-grade test contains 40 such questions. The children are given 6 minutes in which to answer these 40 questions; usually more than half of an eighth-grade class will complete the test. How many questions of the usual school examination could a class of children answer in 6 minutes? Surely, even with brief "fact" questions, less than half that number. But the great saving of time is in the scoring. It is possible for a teacher to score the 40 questions of the history test just referred to at the rate of almost two papers per minute. To realize the saving of time one should think of the weary task which a teacher would have before her, in grading a 40-question examination from a good-sized class. To score the test requires not over 30 minutes for a class of 35 pupils. Examination papers from the same class could hardly be graded in four times that period. And there still remains to be taken into account the fact that the test comes to the teacher with the questions all prepared, and everything ready for use, whereas the examination questions for a written lesson must be carefully made out by her. Surely it would seem not unreasonable to claim that tests may soon lighten, very materially, the drudgery of teaching.

It may be argued, in opposition to the above point, that some of the group scales for measuring general intelligence are hardly easy to score. But it must be remembered that these more elaborate scales really include an extraordinary amount of matter. Thus the army scale Alpha is not altogether easy to score. But it contains a total of 212 questions; and it yields information of very great general significance. When these facts are taken into account, one realizes that the "time-cost" is relatively slight.

To summarize, then, these three facts are of greatest importance to understand, in appreciating the difference between a "test" and the usual school examination: (1) The subject matter of a test is selected with extreme care as matter stressed in the best teaching practice, and of the most practical worth; and the difficulty of this material is determined. (2) Methods of giving, taking, and scoring are very carefully worked out and standardized, so that results are of a clean-cut significance, and wide comparisons from class to class, school to school, or city to city, are possible. (3) A good test is a great saver of time and work, and permits one to obtain a really extraordinary amount of information about a class, school, or school system with an astoundingly small expenditure of time and energy. Surely an educational instrument to be distinguished by such characteristics is neither impractical nor of narrowly limited and only occasional usefulness. And the question at once arises as to what, specifically, tests may do, in contributing to the solution of educational problems.