Evaluating testing, maturation, and gain effects in a pretest-posttest quasi-experimental design

1965

Glass, G.V. (1965). Evaluating testing, maturation, and gain effects in a pretest-posttest quasi-experimental design. American Educational Research Journal, 2, 83-87.

Review of Wachter & Straf, The Future of Meta-analysis

Glass, G.V (1991). Review of Wachter & Straf, The Future of Meta-analysis. Journal of the American Statistical Association, 86(416), 1141-1142.

Who Runs America’s Public Schools

2015 Who Runs America’s Public Schools
Gene V Glass

There was a time when they were run by educators. But all that has changed.

Public education, circa 1945

I was entering Kindergarten in Lincoln, Nebraska, a typical school in a typical town in a typical state. My classmates were all white, as were my teachers, who were all female. The superintendent of schools and principals conferred with teachers about curriculum. Teachers exercised discretion in their choice of books and materials. No one was looking over their shoulders to see that in the first week of February they were teaching Standard #1285 – in fact, there were no standards, and no tests, just books and activities and recess.

At the risk of appearing senile, I cannot fight back the temptation to further compare my experiences of 70 years ago with children’s experiences today. Some might think that Lincoln, Nebraska, was a backward village in the boondocks at the end of WWII. In fact, it was – and still is – a fairly forward-looking town with a very credible university. The college of education at the University of Nebraska was called Teachers College, the name borrowed, I feel certain, from Teachers College at Columbia University. Now Teachers College, Columbia, was the seat of Deweyan progressive education. Many of my teachers would have been trained in an institution that emphasized Dewey’s ideas, and so would have been the very few student teachers to whom I would have been exposed. I recall a nice young woman from the university giving me an individual IQ test when I was in the 4th grade. Nothing much was made of it; she was probably satisfying some requirement for a class.

I don’t wish to romanticize the experiences of my elementary school days, but I think I can summarize their essence in a single comparison. School then was a refuge from life at home and in the neighborhood, which could be harsh and stressful. Today, the tables are turned. For children lucky enough to have loving families, home is a refuge from the stress of life in school. Much has been lost.

Things would soon change. In the mid-1950s, Brown vs Board of Education told the nation that there were two public school systems: one for whites and one for Blacks; and this had to change, because they were not equal. Brown vs Board was perhaps the first expression of the Civil Rights movement, which was to impact the nation in profound ways over the next 50 years.

Brown vs Board of Education signaled the start of trends in demographics and federal policy that were to radically alter the middle class support for public education. In the past, public schools were paid for by property taxes as they are largely today, and no one objected because we were all a nation of upwardly mobile middle class White families ramping up their fertility – in advance of the discovery of oral contraceptives. It is amazing to note – in light of the contemporary whine about taxes – that in 1955, the highest marginal tax rate of persons making over $1,000,000 a year was 81%. That rich people today complain about 28% marginal tax rates says something about whom they imagine their taxes are supporting: not their children or grandchildren, but those of other people.

The Powell Memorandum

At mid-century, business and government operated fairly independently for the most part and with only a few noteworthy exceptions (e.g., Teapot Dome, 1922). But by 1970, all of this was to change in very profound ways.

Lewis F. Powell, started out as an inconspicuous lawyer in a Richmond, Virginia, law firm. His primary occupation in the years leading up to appearance on the national scene was representing the tobacco industry. For a time, he served on the Virginia State Board of Education.

In 1971, the US Chamber of Commerce asked Powell to address the question of the relationship of business to government, a topic that few had written about in the past. Writing in his memo to the Chamber, Powell had this to say: “Business must learn the lesson . . . that political power is necessary; that such power must be assiduously cultivated; and that when necessary, it must be used aggressively and with determination—without embarrassment and without the reluctance which has been so characteristic of American business.”

How strange this strikes us in the 21st Century.

The Powell Memorandum went on to outline multiple avenues of influence by which business could influence government: the creation of the profession of Lobbyist; the creation of think tanks (AEI, Cato Institute, Heritage Foundation, Manhattan Institute, et al.); and multiple means of pressuring universities.

And Lewis Powell did not disappear into the shadows after delivering his memorandum to the Chamber of Commerce; no, indeed. He was appointed to the US Supreme Court by Richard Nixon in 1972 and served there for 15 years.

The Corporations & the Rich

Spurred on by the Powell memorandum, the influence of corporate America on public education began in earnest in the late 1970s and early 1980s. Reagan gave “education reform” a huge boost when he declared in 1983 that K-12 public education was a failure. His remedies? Vouchers and abstinence education.

It was Reagan whose policies set the stage for the next 30 years of change in America. Huge cuts in tax rates for the rich have resulted in a wealth gap in America that is reminiscent of feudal societies in medieval Europe. In 1960, the average CEO’s salary was 20 times the average salary of the worker in the CEO’s company; today, it is several hundred times greater.

While the rich have been getting richer and the poor have been getting poorer over the past 40 years, the poor have been expanding their ranks through heightened fertility and immigration. America today is a bimodal population of young Hispanics & African-Americans and older White people, with an increasingly hollowed out middle class.

I have detailed this evolution in Fertilizers, Pills & Magnetic Strips. And so we find ourselves in the contemporary predicament: Both corporations and a shrinking and aging white middle class are withdrawing their support of public institutions of all types that increasingly are serving other people’s children.

Three Sources of Power Over Public Education: Who Really Runs America’s Public Schools?

There are three major wielders of power that increasingly are determining what goes on in public schools: 1) Corporations whose only goal is increased earnings and higher share value by lowering taxes, as well as corporations who seek income directly by supplying services or even running public schools; 2) super rich dilettantes who think they know better than educators how schools should be run; and 3) the courts. I won’t have much to say here about the courts other than that their force can be considerable, but that it can be blocked for years by politicians and special interests.

Rupert Murdoch (of FOX News & Wall Street Journal) sounded the charge a few years ago when he announced to his fellow capitalists that K-12 public education in American was a half-trillion dollar plum waiting to be picked. Among those who heard the bugle call was the UK publishing giant, Pearson PLC (Public Limited Company).

John Fallon, 52, CEO of Pearson, heads a company with more than $8 billion in revenues that is trying to reinvent education. “It doesn’t matter to us whether our customers are hundreds of thousands of individual students and their parents in China, or thousands of school districts in America,” says Fallon. “What we’re trying to do is the same thing—to help improve learning outcomes.” Fallon waxes philosophical as he shares his view of the future: “This idea of the shift from sort of inputs to outcomes, I think, is one that is now becoming all-pervasive in pretty much every area of public policy… . So why should education be any different?” Such utterances betray nothing so much as a naiveté about public policy.

Pearson is bent on controlling every element of the education process, from teacher qualifications to curriculum to the tests used to evaluate students to the grading of the tests to, increasingly, owning and operating its own learning institutions – like Connections Academies (the online virtual charter school company), which it acquired in the early 2000s.

Consider a recent example of Pearson’s reinventing public education. Pearson landed the contract to redo the GED exam. They decided that the exam needed more thought questions. They rewrote the exam and suddenly the failure rate jumped from about 15% to over 60%.

Pearson is a major pusher of the Common Core, for which it stands ready with the textbooks and the tests to sell to states as soon as they adopt the new standards.

According to some stock analysts, Pearson PLC supplies 60% of the U.S. testing market. Expenditures for assessment by U.S. public schools have grown by 60% (from $1.6 billion to $2.4 billion) from 2010-11 to 2012-13. And the rate of growth may well be accelerating.

The behavior of Pearson PLC in landing big contracts with states has raised some eyebrows. Texas signed a half-billion dollar five-year contract with Pearson for assessment services. The corporation played a very common game of creating a non-profit foundation, completely controlled by the parent corporation, to do its business of enticing potential customers. Education officials in states that were considering how to implement the Common Core State Standards would be treated to luxurious trips to far away places like Brazil and Singapore. New York State AG Schneiderman investigated these junkets and Pearson paid a nearly $8 million fine in two years ago. The corporation folded the non-profit foundation when its mode of operation became widely known.

Corporations and ALEC

Pearson was embarrassed when its non-profit foundation was exposed as a briber of education officials. There are dozens of other corporations who do not wish to have their financial contributions to politicians and public officials out in the open. Thanks to Citizens United – perhaps the most incredible ruling of the US Supreme Court of the last 50 years – corporations now hide their bribes through a non-profit organization known as the American Legislative Exchange Council, ALEC.

ALEC is a non-profit organization funded by large corporations and member dues. Its members are almost exclusively Republican state legislators who pay a nominal fee to join. At ALEC’s regular meetings in attractive places like Florida and southern California, legislators are “educated” on some of the issues currently on the minds of the corporations. These issues deal with things like sentencing of law breakers, sale of tobacco, energy policy, and even education. Drafts of bills are passed out with the expectation that they will be introduced in the legislators’ home states. In 2012, 1,000 bills dealing with education written by ALEC were introduced in state legislatures, and 100 of them were adopted as law. Most dealt with charter schools, tuition tax credits and testing.

ALEC is a major force behind privatizing state prisons and supplying them with prisoners. It causes bills to be introduced providing for mandatory minimum sentences and “three-strikes” sentencing. About 40 states passed versions of ALEC's Truth in Sentencing model bill, which denies parole to prisoners convicted of violent crimes. The U.S. incarcerates a higher percentage of its population than any country in the world.

In 2010, ALEC arranged meetings between the Corrections Corporation of America and Arizona legislators such as Russell Pearce to write Arizona SB 1070, which would ensure CCA's immigrant detention centers a steady supply of “clients.” (Source: Wikipedia, “Private Prisons,” accessed January 28, 2015.) The newly elected Governor of Arizona, Doug Ducey, filed his coming year budget two weeks ago. K-12 public education, which has suffered the deepest cuts percentagewise of any state in the nation since 2008, was cut another $17 million. Private prisons are slated for an increase of $52 million, and a bill to create a new 3,000 in-mate prison operated by Corrections Corporation of America has been introduced into the Arizona legislature. A couple years ago, the Arizona legislature repeal a law that required the state to perform a cost comparison between state-owned and private prisons before making any purchases. It is as though the very letter of the laws are being written by corporation lobbyists.

The Declining Influence of Unions

Concurrent with the changing demographics of the U.S. over the past 60 years has been the declining role of unions in American politics.

There was a time when labor unions were very powerful. My father joined the International Typographical Union as a 18 year-old in 1927 after completing his apprenticeship as a type-setter at the Lincoln Journal. The ITU wielded enormous power until about mid-century because it could shut down newspapers, the primary means of communication at the time. But the revolutionary changes in business of the second half of the 20th Century have decimated traditional labor unions. Outsourcing to labor in the global economy and the role of computers have reduced the price and the power of American labor. My father struck against computers entering the composing room at the newspaper in 1963 and never worked another day at his trade.

The only labor unions left with any clout are those that deal with human services that can neither be outsourced nor computerized. (And the Teamsters shudder at the prospect of driverless trucks.) Foremost among the few remaining labor unions are those that enroll public employees. Consequently, public employee unions have become the target of tax slashing groups. Teachers associations are being recast as collections of incompetent slackers.

The Rise of Elected State Superintendent of Schools

Another manifestation of how control of public education has slipped away from the hands of educators into the hands of politicians is the rise of the phenomenon of elected State superintendents.

In 13 states, the state superintendent of education is an elected position. The increasing politicization of this office has had unfortunate consequences for professional education that will be obvious to anyone who thinks about it even momentarily.

Of the 13 elected State Superintendents, 3 are Democrat (I’m guessing CA, OR and WA), 6 are Republican and 4 are “nonpartisan” (presumably Republicans when you look at the remaining states). This politicizing of a role that historically either did not exist or was occupied by educators represents just one more stage in the removal of control of education by professionals. That politicians are so susceptible to the influence of moneyed interests further raises suspicions that private corporate power in strengthening its grip on public schools.

The 21st Century: The Rise of the Super Rich Ed Reformers

This brings us up to the 21st Century and the entrance of the Superrich Ed Reformers. I shall focus on just two: Bill Gates (actually the Bill & Melinda Gates Foundation) and the Koch Brothers. A by-product of these persons’ ascent to among the richest humans on earth is an overweening attitude that they know more than anyone else about most things. This over-confidence in their own abilities has produced in both of them the Shoe Button Complex.

Warren Buffett (the Sage of Omaha and another of the world’s richest persons) had a partner named Charlie Munger as a long-time partner at Berkshire-Hathaway. Munger’s grandfather had managed to corner the market on shoe buttons back around 1900. The grandfather exercised a virtual monopoly over their production and sale. Emboldened by his business acumen, the old man grew to believe that he not only knew more than anyone else about shoe buttons but that he knew more than anyone else about anything—and he preached and proclaimed at length on such. Munger and Buffett named the syndrome the Shoe Button Complex, and they encountered it frequently in their dealings with successful business practitioners.

Buffett had another insight that he embodied in the principle of the Circle of Competence. He knew what he knew, and he didn’t stray far from what he knew. Step outside your circle of competence, and you are on dangerous ground. Back in the great Internet boom of the 1990s, people repeatedly asked Buffett which Internet stock he was investing in. Buffett reminded them that in the history of the 20th Century the US has known more than 1,000 automobile manufacturers: Essex, Lafayette, Marquette, Ajax, Mercer, etc. How, he asked, am I supposed to pick out which one of these dozens of Internet companies is going to be successful? Instead, reasoning that in the future people are highly likely to wear underwear, he bought stock in Fruit of the Loom.

But Buffett was not foolproof. Twenty-five years ago, he phoned Bill Gates and invited him to Omaha, and he sat him down and told him it was time to start giving away his money. And to underline his advice, he told Bill and Melinda that he was leaving his $50 billion estate to their foundation because they were better able to make good philanthropic decisions than he. It may have been one of the few times Buffett ventured outside his Circle of Competence.

So what has Gates done with his and Buffett’s money? He has spent billions in Africa attempting to lift backward nations and he has spent hundreds of millions attempting to reform K-12 education in America. How has he done? Not particularly well. He recently admitted that his Africa policy to push genetically modified crops on farmers has been a failure.

Bill Gates’s insight into what public education in America needs is obviously not insignificant, given his influence over education policy at all levels today. Gates’s worldview is all about global economic competition. When he was asked to testify to the House Education Committee back in the 1990s, he spent the bulk of his time talking about visas for engineers he wanted to import from India and China to work at Microsoft. It is safe to say that Gates’s conception of young people is that of future employees of transnational corporations, and the purpose of education is to prepare them to make money for the firm.

Gates speaks absolute simplistic nonsense when he says that we can’t improve education if everybody is aiming at different goals and if we don’t have standards to hold educators responsible for attaining and if we don’t have tests to tell us if we have reached those standards. Excuse me, but this is pure poppycock, and if it were not coming out of the mouth of one of the world’s richest men, it would be laughed out of the room. To Gates, America needs the Common Core and “data,” by which he means tests, tests, and more tests. Perhaps he should take a course in qualitative research methods and expand his concept of “data” just a bit.

The Bill and Melinda Gates Foundation has sunk hundreds of millions of dollars into creating and pushing the Common Core State Standards and Value Added Measurement (VAM) of teachers, principals, schools, states and colleges of education, surely.

All that I have to say about the Common Core State Standards is that:

1. They should not be common to all states;
2. They are not a plot by the federal government to take over 15,000 school districts – contrary to the opinion of our the new Arizona State Superintendent of Public Instruction and many TEA partiers;
3. They would if implemented further deskill teachers and contribute to the de-professionalization of teaching.

VAM (evaluating teachers and others by the gains in the students’ test scores) is an invalid monstrosity that has failed in history going back to Ireland in the late 1800s. I can’t imagine that reformers like Bill Gates intend to turn teachers into minimum-age clerks, but their conception of an efficient and effective education system threatens to do just that. “Teachers” in name only who download their curriculum from the state education agency or Pearson PLC, monitor students’ time online, proctor tests, and enter data into data bases so that administrators can make data-driven decisions.

Will Gates soon declare his U.S. education reform policy a failure? He has backed off on testing the Common Core any time soon, and states are backing out of adopting the CCSS under threat of US Department of Education punishment. A nationwide opt-out of standardized testing movement is gaining speed.

The Koch Brothers

Charles and David control the second largest privately owned company in the United States. They are a couple of good old boys from Wichita, Kansas, who, according to Mitch McConnell, essentially own the Republican party. They are major bakers of the TEA Party and their money makes possible the existence of such right-wing think tanks as The Heritage Foundation and the Cato Institute. The Kochs have announced that they plan to spend $900 million on the 2016 election.

The Kochs spent hundreds of millions of dollars in the 2014 mid-term elections, and almost certainly turned one U.S. senatorial race outcome from Democrat (Mark Udall) to Republican (Cory Gardner). The fact that Gardner has repeatedly introduced bills in the Colorado legislature to declare a fertilized egg a person and to outlaw the sale of contraceptives helps to describe the politics of the Koch brothers.

Many have wondered why an issue like the Keystone pipeline is such a hot political topic when it will create an insignificant number of permanent jobs and when its sole function is to move Canadian raw petroleum from the northern oil sands to Texas refineries for eventual sale on the world market. Approving the pipeline has been a do-or-die issue with the Republican Party for a few years now; and quickly after assuming majority power in the Senate in 2015, they pushed through a bill that authorizes the construction and operation of the pipeline. The Kochs are the largest owners of land in the Canada oil sands.

These two fellows are exercising an enormous influence over U.S. politics and they have public education in their sights. For some years now, the Koch brothers have been contributing to the campaigns of school board candidates. As a matter of fact, it has been a strategy of conservative political movements to start promising politicians in school board elections as a try-out for higher office.

In a recent school board election for the Jefferson County (Colorado) schools, two seats were won by very conservative candidates. Their campaigns were financed primarily with Koch brothers’ money. The votes had scarcely been counted when the business of the board erupted into a series of political battles with parents, teachers and community members. Conservatives now held a majority on the board. Proposals were discussed for implementing value-added measurement (VAM) of teachers for salary and tenure purposes, and a superintendent search was favoring an administrator from a neighboring district who had rammed VAM done his teachers association’s throat. But the issue that generated the strongest public reaction was the board’s foray into curriculum. Led by the newly elected board members, an investigation concluded that the Advanced Placement History course textbook did an inadequate job of instilling loyalty and a sense of pride in America’s accomplishments. Changes were ordered. The board passed a resolution that ordered the classes to promote "patriotism and ... the benefits of the free-enterprise system" and that they should not "encourage or condone civil disorder."

For some teachers, students and parents, an attack on the AP History curriculum was a bridge too far. Students reacted first. They walked out of school, or more accurately, they refused to attend. Then they took to the streets with placards, and demonstrated against the intrusion of the board into the AP curriculum. They were eventually joined by parents and teachers. After the walk-out appeared on the local news, the board backed down.

Other Super Rich Ed Reformers

• The Walton Foundation (big backers of charter schools)
• The Broad Foundation (Training administrators to love the market)
• John Fisher (son of the GAP creators investing in charters)
• David Welch (Silicon Valley billionaire who funded the Vergara case)

What Can An Educator Do to Take Back Control of Public Education?

Of course, it is an exaggeration to say that public education today is being run by corporations, politicians, and the super-rich who own them. But it is no exaggeration to point out the growing influence of the forces that seek to reap private gain from this half-trillion dollar sector of the economy.

Is it too late for professional educators to take back control of the schools? Not much. Face it. Politicians are for sale in the Age of Citizens United, and educators are not the ones with the deep pockets.

Diane Ravitch has opined that students – walk outs – and retired teachers are in the best position to resist these forces.

Class Notes: Relationship of Education Policy to Education Research and Social Science

2005

Notes to the Proseminar on the Relationship of
Education Policy to Education Research and Social Science

These notes have two purposes: to disabuse naïve conceptions of the role of research in policy development (if you happen to have any, and perhaps you don’t); to illustrate and conceptualize the actual relationship of social science and educational research to policy.

Since we are working in a graduate program that has training for a career of scholarship and research as its goal, and since it is education policy that is the subject of this scholarly attention, it should not be unexpected that at some point we would ask how these two things come together. There is much more to policy making than doing or finding research that tells one how to make policy—far more. In fact, basing policy on research results—or appearing to do so—is a relatively recent phenomenon. Policies are formulated and adopted for many more reasons than what educational and social science research give. Policies may arise from the mere continuation of traditional ways of doing things; they may come about to satisfy a group of constituents to whom one owes one’s elected or appointed position. They may grow out of the mind of an authority in a way that is completely incomprehensible, even to the leader. But increasingly, policies are being viewed by opinion makers, journalists, intellectuals of various sorts, and even a broader public as not legitimate or not deserving of respect or lacking authority unless they are linked somehow to science. In what follows immediately, I have written down a few ideas on this subject. They may orient your thinking in a way that makes the articles you will read more meaningful. It is also my hope that they will help us understand what went on in the exercise last week when we carried out a simulation of how a policy is influenced by research.

In brief, I believe that research is far too limited in its scope of application to dictate policy formulation. Policy options arise in large part from the self interests of particular groups. No matter how broad and solid the research base, opponents of a policy recommendation that seems to arise from a particular body of research evidence will find ground on which to stand where the evidence is shaky or non-existent, and from there they will oppose the recommendation. If “whole language” instruction appears from all available research to result in greater interest in reading and more elective reading in adulthood, opponents will argue that children so taught will be disadvantaged in learning to read a second language or that no solid measures of comprehension were taken thirty years later in adulthood.

So what is the relationship of research to policy? The answer entails distinctions between such things as codified policy (laws, rules & regulations, by-laws and the like) and policy-in-practice (regularities in the behavior or people and organizations that rise to a particular level of importance—usually because they entail some conflict of interests or values--where we care to talk about them as “policy”).

Policy-in-practice is often shaped by research in ways that are characterized by the following:

• Long lag times between the “scientific discovery” and its effect on practice;
• Transmission of “scientific truths” through popular media, folk knowledge, and personal contact;
• Largely unconscious or unacknowledged acceptance of such truths in the forms of common sense and widely shared metaphors.

Thus do we move from views of children that prevailed in the 19th century—tabula rasa, sinful, responding to corporal punishment, having “mental muscles” needing exercise—to views that prevail in the 21st century—learning through rewards either by operant (Skinnerian) principles or by need reduction, having fragile “self-concepts” that must be nurtured to produce high “self-esteem,” etc.—thanks to Thorndike, Freud, Skinner and the rest. (It may be worth remarking upon at this point that research powerful enough to change prevailing views is rare, resembles what Kuhn has called a “paradigm shift”—although Kuhn quite clearly observed that the social sciences are in a “pre-paradigm state” so that there could be no paradigm shifts in the sense he spoke of them—and is usually rejected as wrong when first presented precisely because it challenges prevailing views. I hasten to add, however, that the vast majority of ideas that challenge the prevailing view are in fact wrong.)

With respect to codified policy, research serves functions of legitimating choices, which it can do on account of its respected place in modern affairs. Research is used rhetorically in policy debates. Not to advance research in support of one’s position is often tantamount to conceding the debate—as if the prosecution put its expert on the stand and the defense had no expert. In rare instances so recondite and far from the public’s ability to understand and in instances where a small group of individuals exercises strict control (“contexts of command” as Cronbach and his associates spoke of them) or in places where the stakes are so low that almost no one cares about them, research may actually determine policy. But even these contexts are surprisingly and increasingly rare. (Research evidence is marshaled against such seemingly “scientific” practices as vaccinating against flu viruses, preventing forest fires, the use of antibiotics, tonsillectomies, and cutting fat out of your diet.)

For the most part, research in education functions in a “context of accommodation” where interests conflict and the stakes are relatively high. There, research virtually never determines policy. Instead, it is used in the adversarial political process to advance one’s cause. Ultimately, the policy will be determined through democratic procedures (direct or representative) because no other way of resolving the policy conflicts works as well. Votes are taken as a way of forcing action and tying off inquiry, which never ends. (As an observer once remarked concerning trials, they end because people get tired of talking; if they were all conducted in writing—as research is—they would never end.)

Research that is advanced in support of codified policy formation is at the other end of the scale from the kind of scientific discoveries earlier referred to that revolutionize prevailing views. They resemble what Kuhn called “normal science”—small investigations that function entirely within the boundaries of well-established knowledge and serve to reinforce prevailing views. Such work is properly regarded less as “scientific” inquiry than as other things: rhetoric, testimony for the importance of a set of ideas or concerns, existence demonstrations (“This is possible.”). [Having written this paragraph in the first draft, now, in the second draft, I have no idea what I was driving at.]

Research does not determine policy in the areas in which we are interested—human services, let’s call them—in large part because such research is open ended, i.e., its concerns are virtually unlimited. Such research lacks a “paradigm” in the strict Kuhnian sense: not in the sense in which the word has come to mean virtually everything and nothing in popular pseudo-intellectual speech, but “paradigm” in the sense that Kuhn used it, meaning having an agreed upon set of concepts, problems, measures and methods. On one of the rare occasions when Kuhn was asked whether educational research had a paradigm, or any recent “paradigm shifts,” he seemed barely to understand the question. In writing (Structure of Scientific Revolutions), he opined that even psychology was in a “pre-paradigm” state. Without boundaries, a body of research supporting policy A can always be said to be missing elements X, Y and Z, that just happen at the moment to be of critical importance to those who dislike the implications of whatever research has been advanced in support of policy A.

This lack of a “paradigm” for social scientific and educational research not only makes them suffer from an inability to limit the number of considerations that can be claimed to bear on any one problem, it also means that there are no guidelines on what problems the research will address. The questions that research addresses do not come from (are not suggested by) theories or conceptual frameworks themselves, but rather reflect the interests of the persons who choose them. For example, administrators in one school district choose to study “the culture of absenteeism” among teachers; in so doing they ignore the possibility that sabbaticals for teachers might be a worthy and productive topic for research. The politics of this situation are almost too obvious to mention.

What are some functions of research in the process of policy formation?

a) Researchers give testimony in the legal sense in defending a particular position in adversarial proceedings, wherever these occur. (There even exists an online journal named Scientific Testimony, http://www.scientific.org/.) Both sides of a policy debate will parade their experts, giving conflicting testimony based on their research. But not to appear and testify is to give up the game to the opposition.

b) Researchers educate (or at least, influence) decision makers by giving them concepts and ways of thinking that are uncommon outside the circles of social scientists who invent and elaborate them. An example: in the early days of the Elementary and Secondary Education Act of 1965 (the first significant program of federal aid to education), the Congressional hearings for reauthorization of the law were organized around the appearance of various special interest groups: the NEA, the AFT, the vocational education lobby, and so forth. Paul Hill, now a education policy professor at the University of Washington, was a highly placed researcher/scholar in the National Institute of Education in the early 1970s. His primary responsibility was for evaluation of Title I of ESEA, the Compensatory Education portion of the law. Through negotiating and prodding and arguing, he succeeded in having the Congressional hearings for reauthorization organized around a set of topics related to compensatory education that reflected the research and evaluation community’s view of what was important: class size, early childhood developmental concerns, teacher training, and the like. No one can point to a study or a piece of research that affected Congress’s decisions about compensatory education; but for a time, the decision makers talked and thought about the problems in ways similar to have the researchers thought about them.

c) Researchers give testimony (in the “religious” sense of a public acknowledgment or witnessing) to the importance of various ideas or concepts simply by merit of involving them in their investigations. (“Maybe this is important, or else why would all those pinheads be talking about it?”)

The effects of exercise on blood lipids and lipoproteins: A meta-analysis of studies

1983

Zung Vu Tran, Arthur Weltman, Gene V Glass, Dale P. Mood. (1983). The effects of exercise on blood lipids and lipoproteins: A meta-analysis of studies. Medicine and Science in Sports and Exercise, 15(5), 393-402.

Die Entwicklung einer Methodologie der Evaluation