Five years ago I wrote a commentary called "Professional Development: How Do We Know If It Works?" for Education Week. In the article, I argued that the only proper measure of success for teacher professional development is whether their students learn more as a result. I pointed out that in the extensive literature on the "best" approaches to improving teachers' skills, there was vanishing little evidence that interventions lead to increased student learning. Mostly this was because the people planning, paying for, and delivering the teacher training were not gathering the data needed to see if it helped students learn more.
Now, however, evidence is beginning to accumulate, at least in the area of mathematics. For a long time, intuition has told us that teachers who know more mathematics must be teaching it better, and indeed, studies have shown that secondary students whose teachers have advanced degrees in mathematics learn a little more than those whose teachers don't. But now (actually, in June 2009)Rolf Blank and Nina de las Alas of the Council of Chief State School Officers have done a meta-analysis of multiple studies that shed some light on what professional development can do to improve student learning.
Blank and de las Alas combed through all the studies they could find on professional development in math and science over a 20-year period. Eventually they found 16 well-designed US-based studies in math and science PD that included control groups (students whose teachers did not get the training) and good outcome measures. Twelve of the 16 studies focused on math. The researchers also identified features common among many of the interventions.
Their findings? In general, the effect of math professional development was positive but modest, with effect sizes averaging .21 for differences in pre-post test growth. That's enough to take an average student from the 50th to the 58th percentile. Effects were greater on test measures that closely matched the training and lower on more distant tests such as state tests.
These successful PD programs tended to share several features. They were focused on math content and how to teach it. The programs were long, averaging over 90 hours, with most of them spread out over six months or more. Included in the PD was a program of follow-up and reinforcement, including support from mentors, coaches, and colleagues to help teachers take what they had learned into the classroom. Effects for elementary teachers tended to be higher than those for secondary teachers.
The report has lots of meaty detail, and is well worth a read for those who want to design and carry out the most effective and cost-effective professional development. But for those who are not statistics nerds, it's good to know that we are finally accumulating rigorous evidence that the money school districts spend on programs to improve teacher skill really can pay off in increased student learning.
Showing posts with label education research. Show all posts
Showing posts with label education research. Show all posts
Saturday, February 26, 2011
Wednesday, February 9, 2011
Learning social skills boosts student achievement
What is social-emotional learning, and what does it have to do with academic achievement?
Fifteen years ago, in the early days of education accountability in Massachusetts, a small elementary school in the poor city of Springfield won recognition for demonstrating unexpectedly high achievement. The principal attributed much of the school's effectiveness to one surprising tactic: the school devoted the first couple of weeks of school each year entirely to building school climate and culture. That is, instead of getting right down to learning, the school wasted time with social fluff. And it worked.
Now University of Chicago professor emeritus Joseph Durlak and colleagues have performed a meta-analysis of 213 social-emotional learning programs involving 270,000 students from kindergarten through high school. Meta-analysis is a method allowing multiple studies to be combined to look for larger patterns. In this case, all the included studies examined the impact of whole-school interventions aimed at teaching kids such skills as how to set goals, resolve conflict, and make sound decisions. All the studies included control groups not exposed to the programs.
Overall, students instructed in social-emotional skills had better outcomes on a whole range of measures. They learned the specific skills taught. They had more positive attitudes toward school and got along better with teachers and peers. They got in trouble less and suffered less anxiety and depression. Perhaps most surprising, their grades and test scores were significantly better than those of students who did not participate in such a program. The difference was 11 percentile points, enough to raise a kid expected to be right in the middle of the class into the top 40 percent.
Not all social-emotional learning programs are equal. Those that worked best were integrated into the school day and taught by a regular classroom teacher, not an outside group. Successful programs contained four elements: They took a specific, step-by-step approach to teaching a sequence of skills; they gave kids opportunity for active practice of those skills; they provided plenty of time for the skills to take hold; and they had specific learning goals.
It makes sense that kids who feel connected to school, know how to get along with their peers and teachers, and attend a school with an ethos of good, cooperative behavior will get in less trouble and learn more. What's great about this study is that it provides strong evidence that social-emotional skills can be taught in the regular classroom, and that taking the time to do so pays off not only in terms of student mental and social well-being but also academically. Good school culture can be built, and it makes a difference.
Fifteen years ago, in the early days of education accountability in Massachusetts, a small elementary school in the poor city of Springfield won recognition for demonstrating unexpectedly high achievement. The principal attributed much of the school's effectiveness to one surprising tactic: the school devoted the first couple of weeks of school each year entirely to building school climate and culture. That is, instead of getting right down to learning, the school wasted time with social fluff. And it worked.
Now University of Chicago professor emeritus Joseph Durlak and colleagues have performed a meta-analysis of 213 social-emotional learning programs involving 270,000 students from kindergarten through high school. Meta-analysis is a method allowing multiple studies to be combined to look for larger patterns. In this case, all the included studies examined the impact of whole-school interventions aimed at teaching kids such skills as how to set goals, resolve conflict, and make sound decisions. All the studies included control groups not exposed to the programs.
Overall, students instructed in social-emotional skills had better outcomes on a whole range of measures. They learned the specific skills taught. They had more positive attitudes toward school and got along better with teachers and peers. They got in trouble less and suffered less anxiety and depression. Perhaps most surprising, their grades and test scores were significantly better than those of students who did not participate in such a program. The difference was 11 percentile points, enough to raise a kid expected to be right in the middle of the class into the top 40 percent.
Not all social-emotional learning programs are equal. Those that worked best were integrated into the school day and taught by a regular classroom teacher, not an outside group. Successful programs contained four elements: They took a specific, step-by-step approach to teaching a sequence of skills; they gave kids opportunity for active practice of those skills; they provided plenty of time for the skills to take hold; and they had specific learning goals.
It makes sense that kids who feel connected to school, know how to get along with their peers and teachers, and attend a school with an ethos of good, cooperative behavior will get in less trouble and learn more. What's great about this study is that it provides strong evidence that social-emotional skills can be taught in the regular classroom, and that taking the time to do so pays off not only in terms of student mental and social well-being but also academically. Good school culture can be built, and it makes a difference.
Saturday, December 18, 2010
High school students test better when paid
When a high school senior sits down to take a test that "doesn't count," how hard does he try? Not as hard as he could, according to a study released yesterday by Teacher's College Record.
For years, policymakers have wondered why 4th and 8th grade NAEP scores, especially in math but also in reading, have been steadily rising, while 12th grade scores remain low and flat. NAEP is the National Assessment of Education Progress, often called the "national report card." Researchers scrutinize the results, politicians bemoan them, and everyone tries to figure out what the scores tell us about school improvement or decline. But for years, thoughtful observers have wondered whether the disappointing twelfth grade scores might reflect student rebelliousness more than student ignorance.
A 4th grader, the reasoning goes, will try her hardest just because her teacher requests it. Most 8th graders will do the same. But by 12th grade, students have taken so many high stakes tests for graduation, for classwork, or for college admission and placement, that a significant number of them may be too jaded - some would say too canny - to expend effort on a test that carries absolutely no individual consequences.
In the past, researchers and test administrators have tried to check for lack of effort by seeing how many questions are left blank or how many sections of a test are obviously answered at random, say by checking all c's. Now, however, a trio of researchers - Henry Braun, Irwin Kirsch, and Kentaro Yamamoto - have done an experiment to try and find out.
The researchers recruited 2600 high school seniors from 59 diverse schools in seven states. They randomized the students into three groups and administered them two sections of the 12th grade reading NAEP. The control group received no incentive. A second group received a $20 gift certificate before the test. The third group, called the "contingent reward group," received a $5 gift certificate at the start of the test, and was also told that two questions would be chosen at random at the end of the test. For each of the two questions students got right, they would receive another $15 gift certificate, for a possible total of $35.
Not surprisingly, the offer of a contingent reward made a difference for these 12th grade students. The difference was equivalent to 5 points on the reading NAEP, which is equivalent to one-fourth of the average 12th grade black-white achievement gap. What's even more interesting is that incentives made the greatest difference for lower-scoring kids.
The researchers' conclusion is that we are systematically underestimating our 12th grade students' reading ability, and the same is very likely true for math, Moreover we are overstating the achievement gap between high and low performers. Given no personal incentive to do well on yet another test, a significant percentage of students make the rational decision not to try all that hard. If we want accurate measures of the knowledge of older students, we need to give the students some convincing reason to do their best.
For years, policymakers have wondered why 4th and 8th grade NAEP scores, especially in math but also in reading, have been steadily rising, while 12th grade scores remain low and flat. NAEP is the National Assessment of Education Progress, often called the "national report card." Researchers scrutinize the results, politicians bemoan them, and everyone tries to figure out what the scores tell us about school improvement or decline. But for years, thoughtful observers have wondered whether the disappointing twelfth grade scores might reflect student rebelliousness more than student ignorance.
A 4th grader, the reasoning goes, will try her hardest just because her teacher requests it. Most 8th graders will do the same. But by 12th grade, students have taken so many high stakes tests for graduation, for classwork, or for college admission and placement, that a significant number of them may be too jaded - some would say too canny - to expend effort on a test that carries absolutely no individual consequences.
In the past, researchers and test administrators have tried to check for lack of effort by seeing how many questions are left blank or how many sections of a test are obviously answered at random, say by checking all c's. Now, however, a trio of researchers - Henry Braun, Irwin Kirsch, and Kentaro Yamamoto - have done an experiment to try and find out.
The researchers recruited 2600 high school seniors from 59 diverse schools in seven states. They randomized the students into three groups and administered them two sections of the 12th grade reading NAEP. The control group received no incentive. A second group received a $20 gift certificate before the test. The third group, called the "contingent reward group," received a $5 gift certificate at the start of the test, and was also told that two questions would be chosen at random at the end of the test. For each of the two questions students got right, they would receive another $15 gift certificate, for a possible total of $35.
Not surprisingly, the offer of a contingent reward made a difference for these 12th grade students. The difference was equivalent to 5 points on the reading NAEP, which is equivalent to one-fourth of the average 12th grade black-white achievement gap. What's even more interesting is that incentives made the greatest difference for lower-scoring kids.
The researchers' conclusion is that we are systematically underestimating our 12th grade students' reading ability, and the same is very likely true for math, Moreover we are overstating the achievement gap between high and low performers. Given no personal incentive to do well on yet another test, a significant percentage of students make the rational decision not to try all that hard. If we want accurate measures of the knowledge of older students, we need to give the students some convincing reason to do their best.
Sunday, December 12, 2010
Vocabulary, Reading, and the Matthew Effect
Reading and vocabulary growth are closely intertwined. The better a child's vocabulary, the better she understands what she reads. The more a child reads, the faster his vocabulary grows. Together, these observations have led to another, known as the "Matthew effect." Coined by a reading researcher, Keith Stanovich, the term "Matthew effect" refers to a point in the Gospel of Matthew which says that the rich get richer and the poor get poorer. In reading, strong readers become ever stronger, while poor readers fall further and further behind.
How does the Matthew effect come about? Struggling readers, those who are having difficulty sounding out or figuring out words, are exposed to less text every day in school. They read simpler books, and they read slowly, laboriously working their way through the sentences. Moreover, they are far less likely than fluent readers to read outside of school. Why would they choose to do something that is so hard and frustrating?
By fourth grade, according to Anderson and Nagy, the difference in reading volume is huge. The average fourth grader (the child at the 50th percentile) spends 14 minutes a day outside of school reading any kind of text. That adds up to reading about 600,000 words a year. The child at the 90th percentile reads 54 minutes a day, or 2.36 million words a year, almost four times as many. Meanwhile, the child at the tenth percentile reads on average only about a minute a day, for a mere 51,000 words a year. Between the 10th and 90th percentile, that's a huge, 50-fold difference in exposure to written words.
The difference matters, because kids who read more gain more knowledge of vocabulary and more background knowledge. The more they read, the more they understand. It's easier for them to learn new material because they have a context which that new material can fit into. They understand pyramids in math because they've read about pyramids in ancient Egypt; they can learn about American history because they aren't scared off by such words as assembly, representation, and compact.
In their research on vocabulary development, Anderson and Nagy look back to early child development in the home. By now it's common wisdom that well-off parents talk to their children more than do parents in poverty or on welfare. In an average hour, working class parents say twice as many words to their children as do welfare parents; professional parents say almost twice as many again. Moreover, the balance of communication among welfare parents is much more tipped toward telling kids what they may not do: 11 prohibitions and 5 affirmations per hour in welfare families, versus 32 affirmations and only 5 prohibitions per hour in professional class families.
The result of this difference in conversational exposure means that children of professionals come to kindergarten knowing on average more than 3 times as many words as children from welfare families.
What does this mean for parents? It means it's important to talk to your young children and to include them in family conversation as much as possible. It also means that older children can still benefit from being read to. If you have a child who dislikes reading and resists picking up a book, it's important, at the same time as you're working to address the reading difficulty, to keep exposing the child to rich vocabulary and rich ideas through reading aloud. Surprising as it may seem, research shows that even a pre-school picture book is richer in unusual words, those that will stretch a child's vocabulary, than is prime time television or the conversation of college graduates with their family and friends.
How does the Matthew effect come about? Struggling readers, those who are having difficulty sounding out or figuring out words, are exposed to less text every day in school. They read simpler books, and they read slowly, laboriously working their way through the sentences. Moreover, they are far less likely than fluent readers to read outside of school. Why would they choose to do something that is so hard and frustrating?
By fourth grade, according to Anderson and Nagy, the difference in reading volume is huge. The average fourth grader (the child at the 50th percentile) spends 14 minutes a day outside of school reading any kind of text. That adds up to reading about 600,000 words a year. The child at the 90th percentile reads 54 minutes a day, or 2.36 million words a year, almost four times as many. Meanwhile, the child at the tenth percentile reads on average only about a minute a day, for a mere 51,000 words a year. Between the 10th and 90th percentile, that's a huge, 50-fold difference in exposure to written words.
The difference matters, because kids who read more gain more knowledge of vocabulary and more background knowledge. The more they read, the more they understand. It's easier for them to learn new material because they have a context which that new material can fit into. They understand pyramids in math because they've read about pyramids in ancient Egypt; they can learn about American history because they aren't scared off by such words as assembly, representation, and compact.
In their research on vocabulary development, Anderson and Nagy look back to early child development in the home. By now it's common wisdom that well-off parents talk to their children more than do parents in poverty or on welfare. In an average hour, working class parents say twice as many words to their children as do welfare parents; professional parents say almost twice as many again. Moreover, the balance of communication among welfare parents is much more tipped toward telling kids what they may not do: 11 prohibitions and 5 affirmations per hour in welfare families, versus 32 affirmations and only 5 prohibitions per hour in professional class families.
The result of this difference in conversational exposure means that children of professionals come to kindergarten knowing on average more than 3 times as many words as children from welfare families.
What does this mean for parents? It means it's important to talk to your young children and to include them in family conversation as much as possible. It also means that older children can still benefit from being read to. If you have a child who dislikes reading and resists picking up a book, it's important, at the same time as you're working to address the reading difficulty, to keep exposing the child to rich vocabulary and rich ideas through reading aloud. Surprising as it may seem, research shows that even a pre-school picture book is richer in unusual words, those that will stretch a child's vocabulary, than is prime time television or the conversation of college graduates with their family and friends.
Subscribe to:
Posts (Atom)
