"Pull Out" in Compensatory Education

"PULL OUT" IN COMPENSATORY EDUCATION
Gene V Glass
Mary Lee Smith
Laboratory of Educational Research
University of Colorado

Paper Prepared for the Office of the Commissioner
U.S. Office of Education, November 1977

ACKNOWLEDGMENTS
We were asked to address the problem that forms the subject of this paper by Marshall Smith and Fritz Edelstein of the Office of the Commissioner, U.S. Office of Education. We have no idea whether or not they had opinions about the "pull out" issue at the outset. If they did, they hid them from us. They encouraged us in every way to examine the question freely and render an independent opinion. That requires commendable nerve, even when the stakes are only $3,000.

We gathered a large number of documents and interviewed many people in just a few weeks. It would not have bean possible to produce this report without the voluntary cooperation of dozens of persons who consented on a moment's notice to be interviewed, or dig up old data, or mail us their reports and papers. The persons who gave us interviews are acknowledged in the Appendix. The staff of System Development Corporation, particularly Ralph Hoepfner and Clarence Bradford, were helpful in performing new data analyses and explaining old ones. Len Cahen of the Far West Laboratory for Educational Research and Development shared several unpublished papers with us. Bob Stonehill of the U.S. Office of Education found reports for us that we wouldn't have obtained otherwise.

SUMMARY
This report examines the research on "pull out," a method or type of school organization for remedial teaching of Title I eligible pupils. Four major issues addressed are: (1) the educational benefits of pulling students out of the daily routine to provide them with compensatory education services; (2) the impact of such action on students; (3) whether a child is better served if he remains in the classroom all day; and (4) alternatives to "pull out" available for providing compensatory assistance to educationally disadvantaged children. Other related issues examined are: the prevalence of "pull out" programs, the benefits or losses resulting from "pull out" programs, teacher contact with and attitudes toward pulled out pupils, financial costs of "pull out" programs, and the potential contribution of "pull out" programs to cultural separatism, racial segregation, or even racism. It is concluded that despite the near universality of pulling Title I eligible pupils out of regular classrooms for compensatory instruction, the procedure has neither academic nor social benefits, may be detrimental, and is used mostly to satisfy Title I regulations. Alternatives to "pull out" are recommended.

"Pull Out" is a method or type of school organization for remedial teaching of Title I eligible pupils. With this plan, Title I eligible pupils are pulled out of regular classes containing both eligible and non-eligible pupils and sent to a different room to receive instruction from a remedial specialist teacher. "Pull Out" has emerged as a prominent feature of compensatory education in the past few years, and it now concerns policy-makers, researchers, and educators alike. This report was written in response to a request from the Office of the Commissioner of Education to examine the research on "pull out." In the course of preparing this opinion, we interviewed about thirty persons in schools, state education agencies, the federal government, universities, and teacher organizations; in addition, we read and, in some instances, reanalyzed data from approximately 150 documents.

The Incidence and Context of "Pull Out"
Roughly 75% of compensatory education pupils receive remedial reading instruction in the "pull out" setting; the comparable figures for mathematics and language are 45% and 41%, respectively. When these figures are corrected to eliminate pupils in 100% Title I eligible classrooms who do not need to be "pulled out," the "pull out" rates in all other classrooms rise to 84% for reading, 54% for mathematics, and 50% for language arts. When one considers further that pupils might be "pulled out" for one of these subjects and not the other, it is plausible to say that in classes not 100% "Title I eligible" the practice of "pull out" for compensatory teaching is nearly universal.

"Pull out" is probably more prevalent in small and medium-sized districts than in large, urban schools. "Pulled out" and "mainstreamed" pupils compensatory education pupils taught in regular classes do not differ in their academic performance before remedial teaching; thus, "pull out" seems not to be prescribed differentially for pupils with varying remedial needs.

The amount of the entire instructional day spent in the "pull out" setting rose from around 5% in 1973-1974 to around 9% in 1974-1975. The percentage of time does not vary by subject taught (reading vs. mathematics) or by grade level (elementary vs. secondary). Although time in the "pull out" setting is a small part of the total instructional day, at Grade 1 it constitutes almost half of the instructional time funded by TitleI. At the elementary school level, one-fifth of the "pulled out" pupils miss regular classroom instruction in the subject for which they are removed from the regular class (i.e., they are "pulled out" of regular reading to receive remedial reading). One-fourth miss social studies; one-seventh miss science. One-third miss no academic subject at all since they are pulled out during study periods in the regular class. (By some chop logic we do not understand, supplanting is not supplanting at all if one supplants science and social studies.)

The "pulled out" pupil has three chances in four of receiving remedial instruction from a remedial subject matter specialist. (The comparable chances for a mainstreamed compensatory education pupil are only one in three.) However, the "specialist" teachers receive very little training for their job (less than ten hours in any one year on the average) and they receive virtually no extra pay (less than 5% more than regular teachers). These data seem to indicate that remedial specialists are distinguished from regular teachers neither by more intensive training nor by the pay they receive. The most cynical assessment of their role and contribution would be that remedial, specialist teachers are merely rechristened regular classroom teachers -- the motive for so designating them being, perhaps, the need to comply with certain Title I regulations.

Finally,"pull out" programs appear to be roughly twice as expensive per pupil as mainstream compensatory programs, probably because of much smaller class size in the former than the latter.

The Effects of "Pull Out"

Experimental evidence is skimpy on the effects of the "pull out" technique per se on pupils' academic progress. A study recently published by the National Institute of Education (September 1977) alleges to show beneficial effects of "pull out" at certain grade levels and in certain subjects and detrimental effects elsewhere. We have examined the data and find little support for these conclusions. The academic gains made by "pulled out and "mainstreamed" compensatory education pupils in the NIE data are virtually identical, differing overall grades and subjects by less than one-quarter month in grade equivalent units. Perhaps a better database for assessing the effects of "pull out" exists in the data files of the evaluation of the Emergency School Aid Act (ESAA) conducted by Systems Development Corporation. There one finds a consistent negative relationship between the percentage of time pupils spend in the "pull out" setting and their math and reading achievement. This relationship was consistent across all grades and subjects; it held true for samples numbering about 10,000 pupils in total. Moreover, the relationship persisted even after more than a dozen background variables were controlled statistically.

A vast body of empirical research on instructional methods and organization is pertinent to the "pull out" problem because the phenomena investigated share various features with the "pull out" technique. Such related topics include the following: (a) ability grouping, (b) mainstreaming the handicapped, (c) racial desegregation, (d) labeling pupils with consequent changes in teachers' expectations of them, and (e) peer tutoring.

Our synopses of the research evidence on these topics are as follows: (a) The research on ability grouping is inconsistent, uninformative, and a battleground for various social ideologies; it was not helpful to us in forming opinions about "pull out." (b) The research on mainstreaming the handicapped was exceedingly skimpy, but the findings of three studies point toward the benefits of integrating EMH, EMR, or emotionally disturbed pupils into the regular classroom. (c) Nearly all research on racial desegregation fails to trace racial mixing at levels lower than the school building. Using the Coleman data, McPartland (1969) assessed the effects on verbal achievement of black pupils having predominantly black classmates instead of white classmates. Although the effect diminished as more background variables were partialed out, the effect of racial segregation at the classroom level was negative in first analyses and never appeared beneficial regardless of how many variables were statistically controlled. (d) Research into the effects of labeling pupils on teachers' behavior toward them and expectations of them proved to be most pertinent and startling. Labeling a pupil "mentally retarded," "intellectually slow," or "academically weak," reduces his academic achievement by one-quarter standard deviation below that of comparable pupils not so labeled. Furthermore, teachers' attention and support for pupils invidiously labeled are reduced by one-third standard deviation below those for comparable unlabeled pupils; and teachers' judgment of labeled pupils' success, motivation, social competence, etc. is reduced by nearly one-half standard deviation. These findings from more than forty experiments indicate that the effects of labeling pupils are large and worrisome. (e) Finally, research on peer tutoring, which presumably could occur less often in the "pull out" programs. Pupils pulled out of regular classrooms would have to receive remarkably effective compensatory programs to offset the potential risks incurred. In our opinion, the "pulled out" pupil is placed in moderate jeopardy of being dysfunctionally labeled, of missing opportunities for peer tutoring and role modeling, and of being segregated from pupils of different ethnic groups.

Historical and Political Context of "Pull Out"

We believe that the "pull out" problem was created by the ESEA Title I regulations and the manner in which they have been interpreted and enforced. To quote one state education department official, "'Pull out' exists for one reason only; because the 'locals' are afraid Big Brother will catch them in a 'supplanting' violation." The practice of pulling Title I eligible pupils out of regular classrooms so that a "specialist" teacher could give them instruction in a separate classroom did not grow out of professional judgment about curriculum or instruction. The history is complex, but nearly any disinterested reading of it leads to the same conclusion: "pull out" is an artifice created by schools at the urging of USOE's minions in state education departments to satisfy regulations concerning "supplementing, not supplanting" and "excess costs." The regulations themselves reflect a philosophy that seems seldom to have been seriously challenged. They are enforced with a nearly obsessive concern that "noneligible" pupils might receive Title I services. Yet the argument can be made that even pupils performing at grade level and above are educationally deprived by merit of attending a school with large concentrations of poor (since such schools attract less qualified teachers, have poorer opportunities for peer tutoring, etc.).

One finds virtually no support for the "pull out" concept among educators or their professional organizations. Teachers worry that pulling pupils out of class creates discontinuities in their schooling and makes coordination of teaching difficult. Others worry that the regular classroom teachers will feel less responsible for pupils whose needs are presumably being met somewhere else by a specialist teacher. The National Education Association regards "pull out" as a minor issue and will merely watch its evolution, being concerned only with keeping pupil-to-teacher ratios low. The "pull out" problem seems to be no one's major concern. But it may well be one of those quiet, inconspicuous matters that count heavily in ways seldom clearly seen.

Conclusions, Observations, and Recommendations

Our work has led us to the following conclusions and observations about the "pull out" technique and several recommendations for dealing with the problems it raises.

1. Pulling Title I eligible pupils out of regular classrooms for compensatory instruction is virtually universal.
2. The "pull out" procedure per se has no clear academic or social benefits and may, in fact, be detrimental to pupils' progress and adjustment to school.
3. The "pull out" procedure is used by schools more to satisfy Title I regulations than because it is judged by teachers to be a sensible and beneficial plan.

We wish to bring the following recommendations to the attention of those persons at all levels who administer Title I programs and who will influence the evolution of compensatory education:

1. The Title I regulations, which now reflect an overweening concern with targeting funds on "eligible" pupils, should be examined.New considerations should be given to the needs of all pupils in poor schools and the integrity of total school programs.
2. Instructional strategies should be designed that would eliminate the invidious labeling of compensatory education pupils and their segregation from classes of "regular" pupils.
3. Teachers, administrators and other persons connected with Title I programs should be informed of the findings of research on the "pull out" method and associated phenomena. 4.Methods should be devised of counteracting the possibly detrimental effects of "pull out" where educators choose to use it or have no reasonable alternatives. Such methods could include means for coordinating instruction across two sites and techniques of teacher observation that lessen the possibility that "pulled out" pupils will be unconsciously neglected in regular classes.

BIBLIOGRAPHY

Bailey, S.K. and Mosher, E.K. (1968). ESEA: The Office of Education Administers a Law . Syracuse, N.Y.: Syracuse University Press.

Billett, R.O. (1932). The administration and supervision of homogeneous grouping. Columbus:The Ohio State University Press.

Borg, W.R. (1966). Ability Grouping in the Public Schools, (2nd ed.). Madison, Wis.: Dembar Educational Research Services, Inc.

Campbell, E.Q. (1965). Structural effects and interpersonal relationships. American Journal of Sociology, 71, 284-289.

Cornell, E.L. (1936). Effects of ability grouping determinable from published studies. In G.M. Whipple (Ed.),The Ability Grouping of Pupils. Yearbook of Nattional Social Studies Education, Part I. Bloomington, Ill.: Public School Publishing Co., pp. 289-304.

Coulson, J.E., et al. (1977). The Third Year of Emergency School Aid Act (ESAA) Implementation. Santa Monica, Calif.: Systems Development Corporation.

Dienemann, Paul F., Donald L. Flynn, and Nabeel Al-Salam. (1974). An Evaluation of the Cost Effectiveness of Alternative Compensatory Reading Programs. Bethesda, MD: RMC Research Corporation.

Eash, M. (1961). Grouping: What have we learned?. Educational leadership, 18 429.

Ekstrom, R. (1961). Experimental studies of homogeneous grouping: A criticalreview. School Review, 69, 216-226.

Esposito, D. (1973). Homogeneous and heterogeneous ability grouping: Principal findings and implications for evaluating and designing more effective educational environments. Review of Educational Research, 43 (2), 163-179.

Findley, W.G. and M.M12\Byran.Ability Grouping: 1970,Status Impact and Alter- natives.Athens, Georgia:University.of Georgia, Centerfor.EducationImprovement, 1971. \

Flynn, D.L, Hass, A.E., & Al-Salam, N.A. (1976). An Evaluation of the Cost Effectiveness of Alternative Compensatory Reading Programs. Bethesda, Maryland: RMC Research Corporation.

Gampel, D.H., Gottlieb, J., and Harrison, R.H. (1974). Comparison of classroom behavior of special-class EMR, integrated EMR, low IQ, and nonretarded children. American Journal of Mental Deficiency, 79, 16-21.

Glass, Gene V, et al. (1970). Data Analysis of the 1968-1969 Survey of Compensatory Education. University of Colorado.

Glass, G.V (1977). Integrating findings: The meta-analysis of research. Review of Research in Education, (in press).

Goldberg, M.L., Passow, A.H., and Justman, J. (1966). The Effects of Ability Grouping. New York: Teachers College Press, Columbia University, 1966.

Goodman, H., Gottlieb, J., and Harrison, R.H.Social acceptance of EMRs integrated into a nongraded elementary school.American Journal of MentalDeficiency, 1972, 76, 412-413.

Kiesling, H.J. Input and Output in California Compensatory Education Projects. Santa Monica, Calif.:Rand Corporation, 1971.

McDill, E.L., Meyers, E.D., and Rigsby, L.C. Sources of Educational Climates in High Schools. Washington, D.C.: U.S. Office of Education Cooperative,Research Report No. 1999, 1966.

McLaughlin, M. W. Evaluation and Reform: The Elementary and Secondary EducationAct of 1965, Title I. Cambridge, Mass.: Ballinger Publishing Co., 1975.

McPartland, J.M. (1969). The relative influence of school desegregation and of classroom desegregation on the academic achievement of ninth-grade Negro students. Journal of Social Issues, 25, 93-102.

Miller, W.S. and Otto, H.J. (1930). Analysis of experimental studies in homogeneous grouping. Journal of Educational Research, 21, 95-102.

National Institute of Education. The Effects of Services on Student Development. Washington, D.C.: The National Institute of Education, U.S. Department of Health, Education and Welfare, September 1977.

Rock, R.T., Jr.A critical study of current practices in ability grouping.Education Research Bulletin.Catholic University of America, Nos. 5 and 6, 1929.

Smith, M.L. Meta-analysis of teacher expectation research. Unpublished paper. Boulder, Colorado: Laboratory of Educational Research, University ofColorado, 1977.

Smith, M.S. "Equality of educational opportunity: The basic findings reconsidered." In Equality of Educational Opportunity, edited by F. Mosteller and D.P. Moynihan. NY: Random House, Inc., 1972.

Tuckman, B.W. and Bierman, M. Beyond Pygmalion: Ability group reassignment and its effects. Paper presented at the Annual Meeting of the American Educational Research Association, New York, 1971.

Vacc, N.A. A study of emotionally disturbed children in regular and special.classes. Exceptional Children, 1968, 197-204.

Appendix

PERSONS INTERVIEWED ON THE "PULL OUT" ISSUE

Dr. Richard Cortright National Education Association
Dr. George Cronk New York Department of Education
Dr. Joy Frechtling National Institute of Education
Dr. Gerald. Freeborn New York Department of Education
Dr. John Garrett Denver Public Schools
Dr. David Gordon California Department of Education
Dr. Susan S. Hartley Northwest Missouri State College
Office of Senator Floyd Haskell Denver, Colorado
Dr. Ralph Hoepfner Systems Development CorpOration
Ms. Linda Jones Colorado Department of Education
Dr. Martin kaufman BEH, U.S. Office of Education
Dr. Michael Kean Philadelphia Public Schools
Dr. Bernard McKenna National Education Association
Dr. Richard Mallory National Education Association
Dr. Robert Mendro Dallas Public Schools
Dr. Lynn Morris Center for the Study of ':valuation, UCLA
Dr. Iris Rothberg National Institute of Education
Ms. Ann Rutherford Denver Public Schools
Dr. Robert Stonehill OPBE, Office of Education
Dr. Gary Toothaker Superintendent of Rifle Public Schools
Dr. Bruce W. Tuckman Rutgers University
Dr. Jean Wellch Systems DevelopMent Corporation
Dr. David E. Wiley CEMREL

Berliner, D. C. & Glass, G. V (2024) Trust but Verify.

2024

Trust But Verify

 

David C. Berliner and Gene V Glass

Arizona State University

 

School improvement programs that work in some places sometimes don’t work elsewhere. School improvement programs that work with some students may not work with others. Programs that appear to have positive effects in the hands of some teachers may fail to produce good effects with other teachers. If this were not the reality of school improvement, we would have found and implemented excellent programs for every state, district, and classroom in the United States by now. But we haven’t, not by a long shot. Instead, we are continually puzzled as we search for high quality education programs that consistently benefit rural white students, or urban black students, or English language learners from hundreds of nations. We also have problems educating the privileged youth of America’s upper-class communities. The education of children who suffer from “affluenza” (Fernandez & Schwartz, 2013) is as disappointing to many educators as is the slow progress of America’s poor students.

 

 It’s past time to lay aside the belief that what works in one setting with one teacher at one time is very likely to work in another setting with another teacher at another time. Education, says our colleague Lenay Dunn (Berliner, Glass, & Associates, 2014), is a complex, intricate endeavor that entails circumstances we can’t control (e.g., family wealth, parents’ education, community support, and special needs of children), influences we can’t easily identify or measure (such as competing school and district initiatives, classroom culture, peer influence, teacher beliefs, and principal leadership), and results we can neither predict nor easily measure (such as resilience, grit, practical intelligence, social intelligence, and creativity). The complex character of teaching children various subjects limits our ability to design programs that function well wherever they are implemented.

 

 However, one must not despair in the face of this reality. Instead, we should feel privileged that we work in a field that is more complex, and thus more challenging, than physics or rocket science. The late, great economist Kenneth Boulding once remarked that if physical systems were as complex as social systems, we would creep hesitantly out of bed each morning, not knowing whether we were about to crash to the floor or float to the ceiling. Educators face the challenges of these unpredictable social systems every day.

 

 Three Obstacles to Transfer

 

 Education is simply too complex to permit the kind of certainty that characterizes the natural sciences, where a finding is a finding is a finding, where whatever was found to be true in Rio de Janeiro can be transferred to Los Angeles, or rural Mississippi, and on rainy as well as sunny days.

 

 Context matters in the social sciences. The context of a study is all of the circumstances that surround the putative causes and effects that the researcher is attempting to study: the locale, the time of year, the socio-economic level of the persons participating in the study. Each of these features of “context” may interact with the relationship of the independent and dependent variables – the cause and the effect – and change the nature of the relationship. Because of their complexity, we may never understand all the interacting influences that make up a particular context, and thus we may never be able to predict when and where a program will and will not work. But it’s more than the complexities of context that limit our confidence in a program’s transferability to a different setting. Three additional problems make it difficult to transfer programs that appear to work to a new and different setting.

 

 The Problem with Findings. First is the problem of estimating the power of the program that we want to import to our school or district. How strong were the original findings? Were the effects strong enough to suggest that we ought to try it elsewhere? Many reports of a successful program or activity present their results as “statistically significant.” But that doesn’t mean much because statistical significance is primarily a reflection of sample size. A pill that works for only one person out of 50 can produce a statistically significant result in a huge clinical trial. Interpreting data also requires knowledge of whether random assignment occurred and whether the investigators were the same people who developed the program under study. It is better to have data about a program’s effects presented as an effect size, which helps us decide whether the program’s effect, despite all the complications in the study’s design, is potentially large enough to be worth pursuing in terms of time, money, and personnel costs. 

 

 But even if the overall effect of a program was impressive, the conditions under which the program did not work are rarely discussed and are not well understood. The famous Tennessee class-size study (Mosteller, 1995), the STAR study, showed impressive overall benefits of smaller classes. Since that study was published, many have argued that major reductions in class size for poor children are likely to have lasting effects on the children’s lives.  But Konstantopoulos (2011) looked within the overall data and noted that results revealed that a large proportion of the school-specific small class effects are positive, while a smaller proportion of the estimates are negative. Although students benefit considerably from being in small classes in many schools, in other schools being in small classes is either not beneficial or is a disadvantage. Small class effects were inconsistent and varied significantly across schools in all grades. (p. 71) 

 

 This is no different a result from what we find in pharmacological studies. A drug may turn out to have an overall average positive effect, and thus is approved by the Food and Drug Administration. Forgotten in the rush to bring the drug to market are the data that show it didn’t work for many in the sample, it harmed some, and among those who showed positive effects were many people who responded because of placebo effects. Pharmacological research is closer to education research than research in the natural sciences is. 

 

  Just as human biological systems vary, and drugs work with some patients and not with others, school and class contexts vary a great deal. Programs like class-size reduction are fine candidates for improving the progress of poor students and the working conditions of teachers, but they may not always work as we hope. Konstantopoulos’s insights into the effects of the class-size study are similar to the advertisements for medicines one hears on television. You hear about how wonderful a drug is—just before the fast talk begins informing you that it may produce blood clots, susceptibility to tuberculosis, increased heart problems, and the like. We eventually learn that overall success is invariably accompanied by many noneffects and quite a few failures. 

 

  But few researchers, and even fewer promoters of programs, do the high quality research that would reveal noneffects, or negative effects for some children, when a given program is in the hands of some teachers and in certain schools. Education research doesn’t provide us with such answers.

 

 The Problem with Replicability. The gold standard of research is often said to be the randomized clinical trial. But we don’t think so. The real standard is a replication of effects by authors who neither produced the original study nor designed the original program. 

 

In medicine, one major study suggested that only 44 percent of the replications of medical research produced supportive data (Makel & Plucker, 2014). Unsuccessful replications most often occurred when the sample size in the original study was small and when randomization was not employed. These are precisely the conditions that describe a great deal of education research. But we don’t have a nonconfirmation problem in education research, as does medicine, because we have an even more serious problem: We don’t even do replication research!  The replication rate for research in our top journals, at well under 1 percent, is frighteningly low. The lack of replications, of course, makes it harder to be confident that a program that works in one location will work in another. 

 

The Problem with Fading Effects.   As teachers change, as student characteristics change, as assessment instruments change, and as school cultures change, a program that seemed successful a few years back may no longer work as it did. Programs need to be monitored for efficacy over time, just as medicines do. Also, ideas that are key to the program of interest may already be in place among the students we want to help, and so bringing the new program in shows little or no effect. 

 

 Lemons, Fuchs, Gilbert, and Fuchs (2014) examined five randomized studies of a supplemental peer-mediated kindergarten reading program involving more than 2,500 students across nine years. They found a dramatic increase in the performance of the control-group students over time. Obviously, if the control groups are doing better on the measures used to evaluate a program’s efficacy, it’s harder for the program to show an effect in a new district or school. The students in the control groups somehow were getting better instruction over time, so the power of the peer-mediated reading program to show its effects got weaker and weaker. We rarely have nuanced or complete data about the students we want to help when we bring in a new program, and this lack of understanding may weaken the effects we finally see. 

 

 The whole idea of “bringing programs to scale” (that is, moving a program from a few schools to many) is also a problem. Control of the contextual complexity in a few classes, or in a school or two, is a lot easier than control of the myriad contextual variables affecting programs in entire districts or states.  

 

 Realistically Optimistic

 

 So things don’t always work as expected. What are school leaders to do? The best they can! Some data are probably better than no data, if collected honestly by individuals who aren’t out to make a lot of money by pushing a program. 

 

 So look at the data. But overselling an idea or program in your own district is a mistake. You’ll need to try it out, probably adapt it to local circumstances, and then it still may not work as intended. But it might. A realistic view of the difficulties that lie in the path to school improvement must not lead to despair. As professionals, we’re expected to seek better ways of educating children. Trying out programs that have been successful elsewhere, designing new programs that fit local circumstances, and attempting to implement what sound like good ideas are characteristic of exemplary leadership. 

 

Three considerations will increase the chances that experimentation will lead to improvement. One is having teacher buy-in. Not much works well if teachers have things imposed on them that they don’t believe in. Second, don’t implement several new programs and ideas simultaneously. Teachers often suffer from overload when new administrators, or state and federal bureaucrats, set out to change too many things too quickly. Finally, make sure new programs and ideas undergo a formative evaluation to find out how things work and how they might be improved. This might entail asking a local evaluator or colleagues from a different school to help with formative and summative assessments of a program. 

 

In 1987, at the signing of a treaty with the Soviet Union, President Reagan remarked, “Trust, but verify.” His advice is our advice: Trust that your colleagues across the United States and around the world have found some good ideas for school improvement that work for them. But verify that their thinking will work for you, too. EL  

 

Postscript

 

Ideas That (May) Travel Well

 

Here are a few pet ideas that we’ve seen work in one place or another that might offer alternative approaches to school improvement:

 

 * Stop looking for answers to local problems in Scandinavia or Asia. The United States is neither Finland nor Singapore, and it’s a lot more complex than either.

 

 * Redraw school attendance areas to achieve socioeconomic balance, and support high-quality early childhood education in those areas.

 

 * Recognize that teachers work in teams and evaluate them accordingly. Make sure the evaluation system has no consequences for teachers associated with student test scores but does include multiple classroom observations and an evaluation of classroom artifacts—tests, papers, projects, and the like.

 

 * Eliminate tracking in grades K–6, and eliminate grade retention (“flunking”) completely.

 

 * Make sure that no school day for students starts earlier than 8:30 a.m.

 

 * Provide libraries staffed with librarians and counseling offices staffed with enough counselors that they can know students personally.

 

 * If you don’t like your reading scores, find ways to have students read more, and forget most other systems that claim to improve reading. There is no "Science of Reading."

 

   References  

 

 Berliner, D. C., Glass, G. V, & Associates. (2014). 50 myths and lies that threaten America’s public schools. New York: Teachers College Press. 

 

 Fernandez, M., & Schwartz, J. (2013, December 13). Teenager’s sentence in fatal drunken-driving case stirs “affluenza” debate. New York Times. Retrieved from www.nytimes .com/2013/12/14/us/teenagers-sentencein- fatal-drunken-driving-case-stirs-affluenza- debate.html

 

 Konstantopoulos, S. (2011). How consistent are class size effects? Evaluation Review, 35(1), 71–92.

 

  Lemons, C. J., Fuchs, D., Gilbert, J. K., & Fuchs, L. S. (2014). Evidence-based practices in a changing world: Reconsidering the counterfactual in education research. Educational Researcher, 43(5), 242–252. 

 

 Makel, M. C., &. Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316.

 

Mosteller, F. (1995). The Tennessee study of class size in the early school grades. Future of Children, 5(2), 113–127.