Qualities related to success in teaching

Front matter

By

Frederic Butterfield Knight

Submitted in Partial Fulfillment of the Requirements

Preface

The only variation from current practice in the organization of the subject matter of this study is the changing of the position of the conclusions. The writer has stated his conclusions at the very beginning of the study. This was done to enable the busy reader to see at a glance the general trend of the study.

As the practical school administrator knows, the gathering of the kind of data upon which this study is based is a matter of some difficulty. It is the kind of data, however, which we must have if science is to aid in the selection of teachers.

Some psychologists may contend that an analysis of teaching rather than the correlation of observable facts with varying amounts of success of actual teachers is the only correct method of determining what tests will distinguish good from poor teachers. No one would deny the value of sagacious insight into any problem of human engineering. So far neither the analysis method nor the correlation method has done very well in practice on the practical job. This study is based on the correlation method. Its shortcomings should not be confused either with the logical soundness or with the practical superiority of test construction on a basis of correlation between test scores and performance.

Frederic B. Knight.

Contents

Conclusions Based on This Study viii

I The History of Teacher Rating 1

II The Method and Data Involved in This Study 5

Chapter I Introduction

This thesis 1 lies in the field of research which is concerned with methods of rating teaching, of determining the significant factors in teaching ability, and of measuring objectively such factors.

This field of research is by no means a virgin one nor is it one of academic interest only. Practical school administrators have no more important and, at present, no more troublesome problems than those which are grouped around the technique of selecting and rating teachers.

Actual isolation of such factors as intellect and temperament, which are indispensable to successful teaching, and the discovery of a method determining whether a prospective teacher possesses the indispensable qualities of a good teacher would be a boon to school administrators.

During the past fifteen years educators and psychologists have given their earnest attention to this series of problems, on which a great deal has been written and on which much research work has been done. Three studies have been selected to show the general development of attempts which have been made to find solutions to different phases of the personal-management problems of our public schools.

meriam's study

Dr. L. L. Meriam, in a research study, Normal School Education and Efficiency in Teaching, published in 1906, Teachers College Contributions to Education, No. 1, Chapter IV, presented data which were used to discover the correlation between teaching efficiency and scholarship in the normal school.

"This is the problem," said Dr. Meriam. "Is the efficient teacher the proficient scholar? To what extent is he so in each of the subjects of the normal-school course? In other words, does the one who stands high among fellow-teachers stand relatively high among fellow-students in the work preparatory to his

1 All data used in this study are on file at Teachers College, Columbia University, New York City.

teaching? Such a study of mental relationships is in itself a study of causes. If it be found a rule that efficiency in teaching follows proficiency in scholarship, then, other things being equal, the latter may be considered a vital contribution to the latter. And this is our present purpose: to discover, so far as possible, what elements enter into the making of a capable teacher. Corollary questions are : To what extent does proficiency in scholarship mean efficiency in teaching? . . ."

In Dr. Meriam's research study an admirable attempt was made to find out the relative teaching ability of a large number (1,185) of normal-school graduates. Equally careful work was done to determine the relative normal-school success of these graduates. Meriam had no accurate measure of teaching efficiency and no reliable measure to equate the amount of success in one school system with the amount of success in another. He encountered the same difficulty in interpreting normal-school marks as measures of scholastic accomplishment. Great statistical ingenuity was shown by Dr. Meriam and his results, by all odds, were the most dependable at the time of the publication of his thesis.

The correlation between normal-school standing and ability or success in the field was found to be so surprisingly low that differences in scholarship among students in the normal schools seemed to bear a negligible relation to future differences in teaching ability. Meriam found that practice teaching during normal-school training was slightly prophetic of the quality of teaching which should be expected after graduation. Examinations containing professional subject-matter did not appear to furnish a significant index of an individual's ability to teach.

The statistical difficulties of Meriam's work should not blind one to its value. It clearly stated the problem of correlating teaching ability with factors which are more or less objective and measurable. It developed a technique of research that was sound in theory. It exercised much influence in taking the problem of teaching efficiency from the field of opinion and discussion and in placing it, where it properly belongs; namely, in the field of research and objective measurement.

Meriam's more important findings are expressed as coefficients of correlation between teaching efficiency and scholarship in normal-school studies. These he reports as follows:

Introduction 3

Correlation between Teaching Ability and Practice Teaching + . 39

Correlation between Teaching Ability and Psychology + . 37

Correlation between Teaching Ability and History and Principles of

Education -f- • 28

Correlation between Teaching Ability and Method Courses +-29

Correlation between Teaching Ability and Academic Courses + . 22

Meriam's data also support his conclusion that, after the first year of teaching, experience, as such, has little if any influence on the improvement of teaching efficiency.

Elliott's study

In 1910 another treatment of this general subject was published which deserves notice. Dr. Edward C. Elliott presented to the second annual convention of city superintendents in Wisconsin "A Tentative Scheme for the Measurement of Teaching Efficiency." This score card has been revised in detail, but the first scheme included all the essential factors. Elliott stated these three propositions which were of more than temporary importance:

"Is it possible to devise and to apply to the teaching process, impersonal, quantitative standards, whereby the relative worth and efficiency of teachers may be determined more justly and with greater precision than under the ordinary practices of the day?

" Does not the effective organization, administration, and supervision of public schools require that the conditions and results of the teachers' work be subjected to measurements of a quantitative rather than a qualitative nature?

"Is it possible for the present generation to make any reliable and satisfactory conclusions concerning the direction and rate of educational progress without standards of value resting upon a quantitative basis?"

The scheme divides teaching efficiency into seven sections and to each section assigns a weight or value. The scheme, in summary, is here reproduced:

I. Physical Efficiency 12 points

II. Moral-nature Efficiency 14 "

III. Administrative Efficiency 10 "

IV. Dynamic Efficiency 24 "

V. Projected Efficiency 6 "

VI. Achieved Efficiency 24 "

VII. Social Efficiency 10 "

Total 100 "

The value of this scheme is that attention is directed to particular traits and that diagnosis of teaching merit is stimulated. The suggested values are, of course, matters of opinion. The assumption that analysis of the teacher and of the judgment of particular qualities, studied in isolation, can be made is highlyquestionable.

boyce's study

In addition to Meriam's study the only other research of extensive nature is that made by A. C. Boyce 1 and published under the title "Methods of Measuring Teachers' Efficiency," Part II of the Fourteenth Y ear-Book of the National Society for Study of Education. Boyce obtained the rating of a great many teachers for general merit and for specific qualities. Then, by a method of correlation, he worked out the relative significance of the qualities. There are many technical improvements in this study over that of Meriam's, but the general procedure is the same.

For fifteen years the teaching profession has been sensitive to problems of recruiting new members. As yet, however, no one knows the exact formula for success in teaching. The complexity of personality and character and the many-sidedness of teaching have continually baffled useful analysis. We know that several measurable traits are not essential to successful teaching, but we do not know what traits must be present in superior instructors. The inspiring advance in the application of psychological methods to the selection of clerks, stenographers, machine operators, and fliers in industry together with similar success in vocational guidance in professional education, such as engineering and dentistry, increases our confidence in the hope that before long psychology will enable school administrators to select teachers with frequency and size of error far smaller than prevails at present.

1 For a discussion of this study, see the last part of Chapter V.

Chapter II Method And Data Involved In This Study

An accurate rating of a sufficiently large number of teachers for general teaching ability must be obtained before any analysis of the significant qualities of teaching is possible. We must know who the good teachers are, who are the poor teachers and who are the fair teachers, before it is worth while to attempt to find out what facts are pertinent in judging their teaching skill. After we get a group of teachers who we know differ among themselves in general teaching ability, by certain amounts or units, then we may proceed, by a method of correlation, to find out what facts about them are of prognostic or diagnostic value.

Such a rating of general teaching ability for 156 grade and high-school teachers who were at work in the public schools of Towns A, B, and C, in Massachusetts, during the school year 1918-1919, has been obtained. There were six groups of teachers. Three of these groups were the grade teachers in Towns A, B, and C, and three were the high-school faculties in these towns. The number of teachers in each group follows:

Town A Town B TownC

Three separate ratings for general ability in teaching were obtained for each group. One rating was secured from the supervisors in each system for their respective teachers. Another was secured by the mutual judgments of the teachers themselves of each group. Another was secured by a consensus of pupils' opinions.

In general the method which was used in deriving the ratings was to have the several judges rate each teacher in the group relative to the other members of the group for the broad quality general ability as a teacher. The theory which underlies this method is this: Where direct measurement in terms of amount is impossible, measurements by relative position in a series may be so controlled that possibly as exact and as true ratings may be obtained as if units of amount had been used. It is assumed that the amount of difference between two teachers who have been thus judged will depend on the ease with which the differences are observed by competent judges.

In using this general method of rating teachers for teaching ability, we have taken it for granted that the good teacher is the one whom competent judges rate as good. We hold throughout this study that the poor teacher is the one whom the judges have rated as poor. These hypotheses will presumably be acceptable to those who are familiar with the theory and practice of social measurements. It may be admitted that one could question the final truth that the opinions of any number of judges, however competent and harmonious they might be, necessarily establish the facts of teaching merit. Thus the really good teacher, it might be held, is the one who gets her pupils on fastest.

To determine how much the progress of pupils is due to any one teacher is not possible by any method or information that is as yet available. Even to measure a pupil's total progress, much less the total progress of a class, is as yet a little venturesome.

It might also be held that the amount of development of character and morality in the pupils is the only test of good teaching and that what others think about the teacher is really irrelevant.

To hold, on the other hand, that competent judgments of teachers, when properly combined, will give a very useful and approximately true rating, as well as probably the best rating method that is now available, is only common sense. This rating method is entirely defensible.

The good lawyer, after all, is the one who is considered a good lawyer by fellow-members of the bar. The poor dentist is the one to whom no other dentist would go or recommend anybody else. The great preacher is the one who attracts visitors. The good teacher is the teacher who is thought to be good.

Where differences in skill among employed people must be determined, judgments in terms of better than the average, poorer than one's associates, and similar expressions, are useful measures of ability. Of course, the final validity of the judgments may be lessened by the presence of constant error in the opinion that it offered, or by the incompetence or paucity of the opinions expressed, or by the failure properly to combine the judgments after they are obtained.

Of the three ratings, — by the teachers themselves, which is labeled "A," by the supervisors, " B," by the pupils " C," — we shall take up first the ratings of the teachers which are indicated by the judgments of their fellow-teachers.

PROCESS OF RATING TEACHERS, BASED UPON THE TEACHERS'

ESTIMATES

Step 1 . Teachers' meetings for each group were called and the teachers were asked to rate each other for general teaching ability, using the relative-position method. The ratings were not in terms of good, fair, poor, because what one teacher might consider good, another teacher who was more critical might consider only fair. This type of difference might run through the series of judgments.

The ratings were not secured in terms of how much below the best teacher you have ever known, or the equivalent expressions, for errors of an obvious nature are bound to creep into any such rating system. The ratings were all given in terms of relative position within the group itself. Thus, when the grade teachers of Town A rated each other, every teacher placed in order of merit all the teachers in the Town A group. The amount of difference between the teachers in the final rating was determined by combining all the judgments of the teachers. Each member of the six groups of teachers, while in a teachers' meeting, rated those in the group to which she belonged in a similar fashion and under similar conditions with the same instructions. The instructions which were given to the teachers follow:

INSTRUCTIONS TO TEACHERS

On this sheet you are requested to give certain ratings of each teacher in the list, including yourself. Please rate every teacher and please be absolutely frank in your ratings. You need not sign your name. Nobody will ever know how you or anybody else rated him. No personal use will ever be made of any of these ratings. They will be used in a purely scientific study to determine the significance of age, education, early interests, etc., etc., for success as a teacher. The names will all be cut off and destroyed as soon as the different items in the inquiry have been numbered to fit the ones to whom they refer. Also, do not feel disturbed because in each respect somebody has to be rated lowest. These ratings are all relative, and the lowest teacher in the group may well be of very great ability. Please be sure to record ratings, even if they seem to you to be little better than mere guesses. The opinions of twenty men give a useful rating, even if any one of the twenty taken alone is almost worthless. 2

On the sheet is a list of the teachers. Choose the teacher of greatest teaching ability and write 1 after his or her name in Column 1. Choose the teacher next below in teaching ability and write 2 after his or her name in Column 1. Write 3 after the name of the one next below in teaching ability, and do so for 4, 5, 6, etc. If two or more seem absolutely equal in teaching ability give them the same rating. 1

After the teachers had read the instructions carefully a few minutes were allowed them for asking any questions that might occur. When it was clear that the teachers understood what was wanted of them, they proceeded with the rating. No names were signed to the rating sheets. It was evident that honest and sincere opinions were expressed. The resultant ratings of each of the six groups of teachers were then examined. Those sheets which were incomplete or did not sufficiently distribute the ratings were discarded. This lack of usable material was not at all great.

The teachers found that rating each other was a method of polite gossip and was evidently more or less enjoyable. For each set of teachers sufficient material was obtained. The spread or range between the poorest and the best teacher was large. In many cases it was as great as the number of teachers involved. The number of useful ratings (97) were distributed as follows :

Grade teachers 30 ratings

Town A Town B Town C

Total 97 "

Step 2. Each set of ratings was then divided into two halves by chance drawings. Each half has been treated separately throughout this study. These halves will be referred to as Group A and Group B. The carrying of two groups makes corrections for attenuation in the correlations and shows also the reliability of the judgments.

A transcript (see Table I) of fifteen of the ratings for general teaching ability of Town A grade teachers has been made. These ratings compose one group (Group B) of the mutual judgments which is treated later to get a single rating of teachers. The

x The complete instructions are given on pp. 46-48.

columns include the complete ratings of fifteen different judges. Thus, by looking at the first column one will see that one judge rated Ah — as the 21st best teacher of the group, Dr — as the 6th best, Sm — as the best, Hi — as the 3rd, El — as the 21st, and so on.

The numbers opposite each teacher's name show the ratings received. Thus, Ah — was rated by one teacher as 21st, by another as 2nd, by another as 1st, by another as 11th, by another as 1st, and so on. In this way, we have approximately 750 ratings of teachers in a group in terms of relative worth. The ratings by fifteen other teachers of the teachers of this group were similarly obtained and transcribed.

We have the judgments of every teacher on every teacher. Occasionally a judge failed to rate some teacher. This was due to the lack of acquaintance with that teacher. This kind of omission is an index of thoughtful estimate, because it indicates the fact that a judge who had no opinion gave no rating rather than record a mere guess. The trustworthiness of these judgments will be discussed later. 1

Step 3. We now have the relative ranking of each teacher in the opinion of fifteen judges. This is a chance half of all of the ratings; namely, Group B. Group A was similarly obtained.

The next step is to combine these two groups of ratings into a single rating. The theory which underlies the procedure requires some explanation. We know several facts about the resultant and combined rating, even before we make it.

First, the final relative arrangement will not be the result of any one individual judgment, since it will be the product of the ratings of fifteen judges. The bias of any single person who has served as a judge will not operate unduly to influence the final result. The fifteen sets of ratings were chosen by chance and chance errors of overestimation or underestimation of any teacher, because of the particular friendship of or dislike for any teacher, by a particular judge, will be offset by opposite chances.

Second, except for a negligible chance in the drawing of the fifteen ratings for Group A or Group B, there will be no constant error, due to the fact that the judges may know some of the teachers very well and others only slightly. No one judge will know all the teachers equally well; but the teachers who are well known

1 See page 17.

TABLE I

A Transcript op Fifteen Ratings for General Teaching Ability Group B, Town A, Elementary Grade Teachers

S3

Ratings by Judges

Ah—

Dr—

Sm—

Hi-

El—

21 1

Co—

Sy-

. .

Pa—

Sp-

Le—

1 It was proper to give two teachers the same rating if their ability seemed equal.

Better-Worse Judgments Based on Table I Read:

1 is judged worse than 2, 7 times with equal or tied votes; 1 is judged worse than 3, 6 times with 1 equal or tied votes, and so on.

abed e

12 — 7 5

3 1 — 6

4 1 — 4

2 3 3 — 4 6

4 2 — 3

5 2 — 2

3 4 1 — 1 7

5 1 — 3

6 3 — 3 h

—

—

_

—

—

—

—

—

451 — 2 893 — 6

6 1 — 2 10 3 — 4

7 3 — 2 12 1 — 5

Columns a, b, e, f refer to teachers by number. Columns c, g, record tied votes. Columns d, h, record worse votes.

to some of the judges will be those who are not so well known to others. Then, too, those teachers who are little known to some judges will be well known to others. Thus, intimate knowledge and lack of knowledge on the part of those who are judging the teachers will be somewhat evenly spread over the whole list.

Third, a fairly minute scale will be possible, because the ratings are spread about as widely as the number of teachers who have been judged. Many of the ratings which were presented by the teachers had a spread of over 40, and the number of teachers who had been judged was 52.

In any scale which is based on relative position, it is well known that absence of personal bias, absence of constant error, and a large number of judges are the chief desiderata. In these ratings such desiderata are present.

THE LOGIC OF THE METHOD

We must have at our command a technique, not only of changing measures of relative position into measures of units of amount, but also of combining incomplete judgments of relative positions into units of amount. Our problem, in simplified form, is to change differences which are noticeable to competent judges into differences of units of amount. A concrete application follows:

If, in the opinion of judges, person A is better than person B, we must find out how much better. Suppose that, in judging A and B, we have ten opinions. If five of the judges think that A is better than B and five think that B is better than A, then we are justified in calling the matter a draw and rule that A and B are equal.

Suppose, however, that six think A better than B, and only four think B better than A, then we are justified in holding that A is better than B. The question now is: By how much is A better than B? The percentage of judges who notice a difference becomes the basis of our procedure.

It is reasonable to suppose that differences which are noticed equally often are equal in amount. We assume that all judgments are of equal value. Further, we arbitrarily define as one unit of difference that difference which 75 per cent of the judges notice.

Thus, if 100 judgments are made comparing A and B and 75 vote A to be better than B, then A is better than B by one unit.

When the data are incomplete, the only thing to do is to compare those judgments which are complete and disregard the judgments which are not paralleled with similar judgments of the person compared.

As an illustration, let us turn back to the judgments of the Town A grade teachers (Table I). In comparing Ah — with Dr — , we shall neglect the fifth judgment of Ah — because it is not paralleled with one of Dr — , but we shall not neglect it in comparing Ah — with El — , for both have ratings given by the same judge.

We also give to the teacher who has been rated the lowest an arbitrary value of one unit. From this as a base we build up the values of the series. It is obvious that the more judgments there are, the better will be the final rating. The more competent the judges are, the more dependable the final rating will be.

These marks will be used as measures only in respect to their differences. Thus there is a difference of 10 units between a teacher rated 15 and one rated 25. But it would not be correct to think of a teacher rated 20 as twice as efficient as one rated 10 or half as good as one rated 40. These quantitative measures of efficiency cannot be compared on a basis of multiplication or division. We are interested in the differences and in no other mathematical relations. Thus, if teachers A, B, C, are rated 10, 20, 30, we know that they vary in ability by equal amounts. For our purposes, it would make no difference, if the measures were 110, 120, 130, or 610, 620, 630, or 750, 760, 770, as long as the quantitative differences should be preserved.

If we know where the true zero of teaching efficiency is, in relation to the values 10, 20, and 30, then, of course, other mathematical relations can be immediately worked out; but we do not know where the true zero point lies.

APPLICATION OF THE METHOD

In computing the final scale, we begin by making a rough approximation of the probable order by inspection or by computing the median rank of each teacher for her fifteen ratings. From an inspection of the Town A grade ratings, it is easily seen that Ah — will be better than Co — , for example, since Ah — 's median rating is 6, while Co — 's is 11 +. The rough approximation is then refined. This approximation is done by comparing each teacher with the one next above and next below and by continuing the process for several places each way. Usually three places will be sufficient. This refinement is continued by finding the percentages of the judgments which are in favor of each teacher and the percentages of the judgments which rate numerically lower each teacher in comparison with those near her. In every case only those judgments are used in which both teachers who are compared are also rated. The approximate arrangement, in some cases, will be wrong. Then the teachers must be shifted in order. By using the unit values which have been calculated from the table that corresponds to the percentage differences, a scale of amount of difference between the teachers can be built up.

In actual procedure it is better to begin with the worst teacher and work up. A procedure of trial and success, with frequent shiftings back and forth, will be found more economical of time and patience than a more complicated method.

Referring back to Table I, we find that in comparing Ah — with Dr — the fifth rating must be disregarded, as it is incomplete. In seven cases Ah — is higher numerically or worse actually than Dr — . There are twelve usable judgments in all. This gives 7 out of 12 votes, as it were, against Ah — . The percentage is then found and its corresponding unit value. It is well to determine the percentage differences, not only between a teacher and the next one to her, but also for three teachers away. In this way any individual, if rightly placed, will be above not only the one next lower, but also above the three next in order. Where mistakes in order occur, then there must be some shifting.

Two exceptions should be noted: Tie ratings are split, and when there is a 100 per cent agreement that one teacher is better than the next, then there is theoretically an infinite, or at least an unknown, amount of difference between them. In this case commonsense is as good a guide as any. Either the statistician can make two or three indirect comparisons through other teachers' ranks in comparison with the two in question, or he can assign a value to 100 per cent a little larger than that assigned to 99 per cent. The latter procedure was followed here as the amount of difference between a 99 per cent to 1 per cent vote is 3.45 units. We arbitrarily assign 4.00 to 100 per cent to per cent comparison.

While no doubt the explanation seems complex, the procedure, if followed as described above, will readily yield a well-made scale. Turning back to the transcript of ratings (see Table I), which were made by the Town A teachers, we find that the "Better-Worse" columns give in detail the comparisons in judgments. It reads: 1 is worse than 2 (with no equal or tied votes) 7 times; 1 is worse than 3 (with one equal or tied vote) 6 times. Of course, when the worse votes are less than the better, the table is used in the same way.