DR. RAUDENBUSH: Thanks very much, Rena. That was a terrific summary and people have read the paper. So, I don't think I should talk very long at all.

I just want to emphasize a couple of points. It seems to me that over the last I don't know how many years, 30 years, 40 years or whatever we have had in policy research in education a kind of causal model implicitly that we have been operating from that policy makers are going to do something over here and then outcomes are going to come out over here, like achievement and stuff like that, and so for example, one of the big things policy makers might do is give more resources, so, reduce class size, more per pupil funding, teachers with better qualifications, better facilities and we do those things over here and then out pops on this side higher student achievement.

More recently we have a model that is not sort of putting resources in. It is putting incentives in. It is like let us do accountability. Let us get everybody motivated by sort of putting them on a reinforcement schedule if you will where they are going to know how well their schools are doing and if they are not doing well create some strong incentives for them to do better, kind of put this accountability system. People are going to be highly motivated. It is going to pop out as higher achievement, the other one being governance. Let us shift to governance structure. Let us get rid of some of the constraints that school districts put on schools by having choice and having charters and private schools again so it is kind of a governance shift and out here comes achievement.

All of those reforms assume that people who are properly motivated and properly resourced will know what to do, and it is interesting if you think about medicine, nobody says, gee, let us give, I mean they do say, yes, let us give the medical establishment a lot of money and they do say let us give them a lot of incentives. I mean let us say that they could maybe give them more incentives to keep people alive and keep people well and stuff like that but nobody says that would be sufficient to make good outcomes, right?

Nobody would say that and nobody would have a policy agenda that says, "What is the effect of per patient spending on the death rate?" or the heart of research in medicine is clinical research on how to do clinical practice better, how to have better medicine, how to have better surgery, what should be done with the resources and the motivation, and I think that that is what we need to do here and I guess in education I think that has kind of begun to shape up as an understanding.

I am in favor of studying the policy options of the resources, the incentives, the governance. That is really important to do, but we really have to have a lot better knowledge on how people in classrooms can do if properly motivated. I think people are pretty well motivated at this point with No Child Left Behind, and maybe not so well resourced. But you know with the appropriate resources and incentives that they could really know what to do, but our knowledge of how to do instruction and how to organize instruction is really extraordinarily weak, and I think that turns out to be kind of research agenda and to me, at least, it is in that context that randomized clinical trials as in medicine to try to test and get new evidence on the effects of interventions really makes a lot of sense. But then the main point of the paper really is if we take that as kind of the research agenda, all right, then what are the implications? Like how would you really put together a comprehensive agenda in research to improve teaching and learning in classrooms? It would have, in my view, randomized experiments as being the central role, but it would have a lot of other things.

I mean that enterprise relies on a lot of knowledge about what kids know and can do, which kinds of kids know different things. We have got to have, you know, we wouldn't know about achievement gaps between black and white students over the last 40 years if it hadn't been for NAEP. We need to put together outcome measures that reflect the goals that we are generating and the new ideas about teaching about how to teach math and science for example. We can't necessarily just pull these off the shelf.

There is a lot of quantitative and qualitative research that is required to have those kinds of tests and we need to identify good interventions. That takes a lot of different kinds of research on different scales, too. It is a mixed scale research. It is not just a mixed method research.

Some people are looking closely at expert teachers and describing what they do. Other people are trying to design interventions that could enable other people to do what those people do and do them on a larger scale. So, there is a mixed scale aspect to this.

So, really what I am trying to put on the table is I argue and I obviously feel that this research agenda on intervening in classrooms and schools is the critical thing we need to be doing, but then thinking about what are the pieces of that and how do the different methodologies contribute was really the major purpose of the paper.