Author Archives: Paul Horwitz

Early start in educational research

New York Times Clipping

Paul Horwitz, senior scientist, got his start in research earlier than most — when he was three! We’ve enjoyed his stories for many years. This one was too good not to share. One day at lunch we decided to follow up on his memories and dig a little deeper. We contacted Lindsey Wyckoff at Bank Street College, who sent us this story from their archives. Here is Paul’s story:

It’s July 1942. Hitler’s armies have conquered most of continental Europe and are about to unleash their fury on the Russian city of Stalingrad. England has survived the “blitz” but thousands of frantic British parents have allowed their children to be evacuated, some as far away as Canada. In New York the Bank Street Nursery School, under the auspices of the Office of Civilian Defense, has embarked on an ambitious experiment. Forty-five preschool children, ages two to five, will be “evacuated” for six weeks to Lake Waneta in upstate New York in order to evaluate whether the trauma of being separated from their parents outweighs the risk of exposing them to possible attack.

I was one of those children.

I was three and a half, far too young to understand what was happening to me, much less why, but the weeks I spent at “camp” that year are among my earliest memories. And the memories, by and large, are good ones.

I remember being introduced to a special kind of photosensitive paper that could record the silhouettes of objects placed upon it. I remember kicking my legs in shallow water, thinking guiltily that I had tricked my parents into believing I could swim. I have a hazy memory of a newsreel crew with a huge camera that moved back and forth on wheels.

I have no recollection of the battery of psychological tests that must have been run on me, though I do remember my answer to one question: in a race would you rather be first or last? (I chose last, on the basis that that way I wouldn’t always be looking behind me to see whether someone was catching up.)

I have since learned that the experiment was a success: given proper care, including cuddle time as well as meals, young children proved unexpectedly resilient. So no permanent damage was done, though I very much doubt that one could attempt this kind of thing today. In the end, as we know now, no evacuation of New York or any other American city was deemed necessary. The broad Atlantic and the absence of aircraft carriers from the German fleet offered protection enough in that long ago time. But today, as we learn to cope with sporadic and unpredictable violence resulting from a protracted “war on terror,” it is perhaps instructive to remember that we have survived much worse.

How not to Learn from Games

They’re the in thing, especially for teaching science. Everyone, it seems, is fascinated by the potential of educational games. They’re interactive and “multimedia,” they can adapt to individual students, they promote “authentic learning.” In short, they match the outsize expectations of a digital world. They’re definitely cool, but do they teach, and if so, what do they teach?

Full disclosure: I am an enthusiastic proponent of educational games. I created one called “ThinkerTools” so long ago that it ran on a Commodore 64 computer and had to be programmed in machine language to make it run fast enough. And, yes, I have no doubt that kids learn from such games. But do they learn what we think they’re learning? And how would we know if they were? Is it sufficient that they get better at the game? Surely not, else chess masters would be good at logic, and athletes would be physicists.

It is tempting to imagine that we can design educational games so cleverly that it would be impossible for a student to get good at the game without acquiring a deep understanding of whatever it is the game is trying to teach. Unfortunately, it doesn’t always work that way, as I learned from my experience with another educational game called GenScope.

GenScope was a multi-level genetics game. It linked processes at all different levels, from molecules to ecosystems, and we used it to create a bunch of engaging challenges for students. Our species of choice was dragons. We would show a dragon’s chromosomes, for instance, and ask students to figure out how to change its genes to make the dragon breathe fire. Later on, we would challenge them to breed a strain of blue dragons, or try to find two parent dragons that could only have two-legged offspring (hint: neither parent can have two legs).

We used the GenScope games in several high schools. We compared students who had used the games to others who had learned genetics by conventional means. To do this we designed a clever test that assessed precisely the reasoning skills we were trying to teach—and that we naively assumed were necessary to succeed at the games. Each time we did this, we found that the GenScope classes did no better on the test than the control group. Sometimes they did worse!

In the jargon of the trade the Holy Grail is “transfer,” and we weren’t getting much. Knowledge gained in one context is often difficult for the novice to apply to another one, even though to an expert the two situations appear very much alike.

To us, the researchers, the genetic principles behind the GenScope games were obvious, and their relevance to the questions on the test equally so. Clearly, that was not the case for the students, who became expert GenScope players but failed to apply what they learned to genetics.

There are ways around this impasse, of course, and I will describe a few in a future blog post. For the moment, though, let’s just keep in mind: there are lots of ways of getting good at an educational game. Only one of them involves learning what the game is supposed to teach.

What would it take to disprove Intelligent Design?

Scientific theories differ from other belief systems in that they are testable; in other words, they can be disproved. Imagine reading, for instance, any of the following headlines:

  • “Modern Chicken Fossil Found Side By Side with Dinosaur Bones”
  • “Chimpanzee DNA Radically Different From Human”
  • “New Data Shows Earth Only 10,000 [or 100,000 or 10,000,000] Years Old”

What do you think would happen to the theory of Evolution if any of those things occurred (assuming, of course, that the observations were replicated and confirmed)? It would certainly have to be radically modified, and might have to be rejected entirely, because according to the theory it just can’t happen that chickens and dinosaurs ever co-existed. And if human and ape DNA were found to differ by more than a few percent it would be very difficult, if not impossible, to reconcile that with present-day views of how these creatures evolved (relatively recently, from a common ancestor). And if the earth were really “only” ten million years old (much less ten thousand!) there wouldn’t have been nearly enough time for living cells, much less human beings, to have evolved.

In contrast, can you think of any way to disprove the theory of Intelligent Design? I used to think I could. Why not, I thought, look for imperfections in the design, instances where certain creatures seem less than ideally designed for their purpose. (As it happens, there are many examples of such suboptimal design.) But that approach doesn’t work. All the inefficiencies can ever prove is that the designer “works in mysterious ways,” or has a different aesthetic from ours about such things. So what appear to be botched designs may tell us something about the designer but they do not discredit the theory of Intelligent Design itself.

Intelligent Design doesn’t make predictions–other than the trivial one that living creatures should look as though they were designed. But that’s not a prediction; it’s just an explanation for a set of observations. It’s kind of like saying “Lightning looks as though Thor is throwing thunderbolts at us, therefore that’s what it must be.”

The Thor model of lightning is unscientific not because it’s wrong but because it’s untestable. There is no way, short of looking around for Thor and not finding him (and he could, after all, be hiding somewhere), of checking out the theory. Like Intelligent Design, it makes no predictions and, therefore, cannot be disproved. In contrast, the theory that lightning is caused by electric currents, while considerably harder to understand and at first blush a lot less plausible than the Thor model, predicts, among other things, that if Benjamin Franklin flies that kite in a thunderstorm one more time he’s liable to get fried.

Science is all too often taught as though it were merely a collection of facts. What we should be teaching is the process by which we have come to trust those facts, what evidence backs them up, and, most important, what new information could get us to change our minds. We need to teach kids that the hallmark of every scientific theory is that in addition to explaining known data it makes predictions about data that hasn’t been seen yet. Which means that every scientific theory is in constant danger of being disproved if those predictions fail to come true.

Until the Intelligent Design proponents can point to some finding–anything!– that might in future cause them to revise or abandon their theory, that theory is no more scientific than the once widespread belief that the plague was God’s punishment for our sins. If we allow such theories to be treated as science we might as well go back to curing disease by whipping each other to atone for those sins.

Evolution Readiness Progressions

The basic concepts of evolutionary theory are contained in the National Science Education Standards (National Research Council, Washington, DC, 1966) as well as those of the various states. For example, in the table below we show the alignment between the “big ideas” of evolution and the science standards for three states: Massachusetts, Missouri, and Texas. The “learning progressions” in the second column are adapted from the Atlas of Scientific Literacy (American Association for the Advancement of Science, 2007), while the quotes in the third column are from the Massachusetts Science Framework; we also index in that column the corresponding standards from the Missouri “Show-Me” Standards and the Texas Essential Knowledge and Skills Standards.

Beginner Level

Big Idea Learning Progression MA Science Framework
Basic needs of organisms Plants and animals need air and water; plants also need light and nutrients; animals also need food and shelter. “Identify the ways in which an organism’s habitat provides for its basic needs.” See also MO Science K-4: VII.B.2, TEKS Grade 4:5.A&B
Organisms and their environment For any particular environment, some kinds of organisms survive well, some less well, and some cannot survive at all. “Identify the structures in plants and animals that enable them to survive in an environment.” See also MO Science K-4: VII.A.2, TEKS Grade 4:8.A
Interspecific differences Plants and animals have different life cycles that include being born, developing into adults, reproducing, and dying. “Classify plants/animals according the physical characteristics that they share.” See also MO Science K-4: VII.C.1
Basic needs of species Groups of organisms can survive even though every individual in the group eventually dies. “Give examples of how changes in the environment have caused some organisms to die.” See also TEKS Grade 4:8.B
Interactions between species Organisms with similar needs compete with each one another for resources. “Investigate how invasive species out-compete native ones.” See also MO Science K-4: VII.A.2, TEKS Grade 4:8.B
Intra-specific differences Individuals of the same species may differ. “Observe differences between organisms.” See also MO Science K-4: VII.E.2, TEKS Grade 4:8.A
Heritability of traits Offspring are usually very much, but not exactly, like their parents. “Differentiate between inherited and other characteristics.” See also MO Science K-4: VII.D.2, TEKS Grade 4:8.C

Intermediate Level

Big Idea Learning Progression State Learning Standards
Basic needs of species For a species to survive, the individual organisms in it must reproduce fast enough to replace the ones that die out. “Describe how organisms meet their needs by using behaviors in response to stimuli received from the environment.” See also MO Science 5-8: VII.C.1&2, TEKS Grade 5:5.A
Interactions between species Every animal species depends on another species, plant or animal, for food. “Give examples of how organisms can cause changes in their environment to ensure their survival.” See also MO Science 5-8: VII.E.2, TEKS Grade 5:5.B
Intra-specific differences Differences between individuals in a species may give some an advantage in surviving and reproducing. “Give examples of how inherited characteristics may change over time as adaptations to changes in the environment that enables organisms to survive.” See also MO Science 5-8: VII.E.1&3, TEKS Grade 5:10.B
Heritability of traits Some traits of organisms are inherited from their parents; others are learned or acquired. “Recognize that every organism requires a set of instructions that specifies its traits.” See also TEKS Grade 5:10.A

Advanced Level

Big Idea Learning Progression State Learning Standards
Heritability of traits Heritable characteristics can affect the likelihood that an organism will survive and reproduce “Relate the extinction of species to a mismatch of adaptation and the environment.” See also MO Science 5-8: VII.E.4, TEKS Grade 7.10.C
Genetics Heritable traits are transmitted from parents to offspring via different forms of genes, called alleles. “Recognize that hereditary information is contained in genes located in the chromosomes of each cell.” See also TEKS Grade 7.10.C
Survival of fittest individuals in an ecosystem Offspring of advantaged individuals are more likely than others to survive and reproduce, increasing the proportion of organisms that have advantageous traits. “Recognize that biological evolution accounts for the diversity of species developed through gradual processes over many generations.” MO Science 5-8: VII.E.4, TEKS Grade 7.10.B