MR. REDISH: Our time is just about out. So, I am going to try to be very brief. Because I have a slightly different perspective, being of a somewhat different community than everyone else here, I am going to take a more general view.

I was a theoretical physicist for 25 years and, for the past dozen years, I have been doing education research from within the physics department.

That specifically makes me sensitive to the interdisciplinary quality of education research, something that, in fact, very much attracted me to the field. I enjoy it very much.

People in my group use everything from cognitive science, fundamental statistics, ethnology, neuroscience, sociolinguistics, and auto mechanics.

This makes it very clear to me that scientific standards of evidence are not unique but, rather, are discipline specific.

Peer review I see as playing a very critical dynamical role in the creation of a scientific community, and of a discipline.

There really is no area in education which at this point I would call a scientific discipline. They are all evolving communities, especially because there has been so much of an explosion of knowledge and relevant information.

So, peer review is not just about finding scientific merit in a particular area. It is about defining it and creating it.

So, peer review helps establish membership, language through shibboleths. This has been particularly clear to me as a physicist trying to get funding in education research.

I know that, if a physicist was going to read my proposal, there are certain things I must not say. If an education specialist is going to read my proposal, there are certain things I must say. If both those people are going to be on the committee, it is like walking a tightrope.

So, peer review has a lot to do with the establishment of a disciplinary paradigm and the development of the epistemology of the discipline.

More than that, in the reviews, when the committee of visitors to an NSF program sits down and evaluates the program, they not only evaluate whether the quality of the proposal is high and appropriate, and that inappropriate things have been rejected, but they also evaluate the grants funded from the point of view of balance, fairness and matching community goals.

So, the whole evaluation criteria are not only the research quality of the proposals. You might need to fund a certain number of junior people in order to keep the field having quality in the future. So, it is not just about quality now. It is about quality of the field overall.

In addition, when it comes to funding agencies, it is also about a negotiation between the community at large, as funding gets by the science, and the discipline that wants to do it.

In education, one of the things I have learned is that it is very hard for the research to be non-political. As a physical scientist, this was something difficult for me to deal with, but I think I understand it.

There are dangers to the peer reviews. As a community begins to try to form itself, there is a danger that it narrows too quickly, that it becomes closed too early in the history of the discipline, so that it locks itself into a paradigm, and the reviews become automatic and not sense making.

You can become too driven by immediate needs, lose perspective. I guess the most important thing that I think needs to be kept in mind, in finding an overall approach to peer review, is to remember the value of pluralism.

I learned many years ago from Elliott Montsol(?), a statistical physicist who studied ecology, a fundamental ecological principle and that is, a species which relies on multiple sources of food maintains a much more stable population than one which specializes in one.

So, as these communities grow, it is extremely important for us to maintain breadth and flexibility and openness, and not to become too narrow, precise and finite.