DR. JONES: Well, good afternoon. I was sitting her wondering to myself which is better, to be the last on a program or to be the first after lunch, I haven’t decided yet. I too would like to express my appreciation to the committee and to the National Research Council for convening this group and this focus on this important area. And at a time that I think it’s an exciting time to be involved in education because I don’t know a time when there have been all forces focused on it. I’ve been asked to talk about the implications for traditionally under served populations and I’m responding to, I picked five questions, also, I also started off as a math teacher, so I had to do my own math.

The first one has to do, incidentally I conducted a non-randomized, non-representative sampling of some people to get some additional ideas in addition to what I have read and experienced and been involved with myself. How under served populations perceive randomized field trials, generally, and the push for them in the current scientifically based research initiative specifically. And what I’m going to share with you is just some comments that were mine and others and those that I’ve gleaned in my work across the country.

First they say that randomized research designs involve denying beneficial services or interventions to some students, we’ve heard this raised a number of times today and whether you agree or not or think that it’s something that is appropriate to feel or to think that is something that is there. And such things are suggested as that you must be careful and see that the control group gets the services later. One of the studies today talked about doing that, or some other things that could be seen as one’s addressing equity issues.

Decisions of what variables receive randomization is subjective and this speaks to the issue of what’s going to be looked at, what’s going to be randomized, and it really gets at something that is underneath. Some of the comments that were made and the perceptions that are there, that deals with a lack of trust with the research and evaluation community from those traditionally under served groups. Some of it probably comes from some who feel burned, some comes from not having been involved and not really knowing what is going on, just the unknown. And some comes from the issue of lack of diversity quite frankly in the education research and evaluation community, of seeing it be so different from the populations that are being looked at and studied.

There’s no such thing they said as a truly accurate reliable and valid randomized experiment in education in the real world. We’ve heard today about some of the real world or enacted research and designs as opposed to those that are intended. They’re always other outside influences in life. One may think that she or he has randomized groups but political and other issues may make it impossible to insert a randomly selected control group or maintain the control group as a control group.

The complexity of issues surrounding achievement in diverse settings requires a comprehensive intervention and research approach. The present focus of RFT’s is too narrow and constraining many think. RFT’s are not the end all and be all, there’s a feeling that they’re being portrayed now as the answer to everything and there’s a strong feeling as we’ve heard today in recognition that it depends on the question that you’re asking, it depends sometimes on where you are in your development of knowledge in the area, and it depends on the circumstances, the environment and so on like that. Perhaps there are many more areas where randomized design or field trials would be appropriate but certainly not all. To me I don’t know how anything would ever get to just either/or, that it is in fact depending upon the question and the time.

It is a fact that money for research and evaluation is always limited, narrow focus of RFT’s will decrease the availability of resources for other important research it is felt.

There’s an overall feeling that poor minority groups are the most evaluated and researched populations while they have no input into the process. While such national groups as the Evaluation Association for instance are viewing their standards, practices, and guiding principles to incorporate issues surrounding contextual validity and relevance in order to enable evaluators to plan and conduct contextually responsive evaluations some fear that RFT efforts and focus are moving in the opposite direction, decontextualization is the fear.

Other perceptions. There’s a perception that more effective tools and strategies for data collection are needed and especially in diverse settings. There’s a perception that RFT focus is an opposition to the view that culture free evaluations are not the most accurate or helpful in the diverse society of the 21^st century in the U.S.

Question two, reflections. What are the benefits and drawbacks of the trends toward RFT’s for traditionally under served groups? First focus on RFT’s to the exclusion of other methods and designs decreases the likelihood of understanding not only that there are differences in outcomes but why and how and for what subsets of students, we’ve heard this repeated and underscored over and over today. Qualitative methods contribute richness and context to interpret outcomes when presented along with, not in place of, quantitative methods.

This next one is one that would certainly be in sync with issues surrounding urban issues as a number of these there’s overlap. Because of the high mobility rates of students and teachers in schools and districts heavily populated with traditionally under served groups the sample size needed will be extremely large because of the attrition rates over time. And also the time needed to implement full treatments can be extensive, especially when students have been without resources for so long. There may not be time for the full treatment to take place, randomization does not address some of these issues.

Causal relationships can be drawn if significant differences in outcomes are found, and of course that’s a good thing. Confounding variables could include differences in teacher confidence, quality and expectations, I think there was concern raised about one of the studies that was described today as to whether that would be a possible topic. Research on randomized field trials is extremely costly and time consuming, and that is not a small item to consider and we’ve heard this over and over again.

The third question has to do with the kinds of capacity that are required for traditionally under served populations to participate in and use the findings of randomized field trials. Institutes and other non-traditional learning opportunities must be funded with stipends for students and faculty, these institutes must be offered in settings where those involved feel supported and encouraged. HBCU’s and Hispanic serving institutions are likely places to offer some of these institutes. Graduate courses in experimental and quasi experimental research methodology with stipends targeted at members of under served groups is also something to be considered.

There’s a need for greatly improved dissemination system. Reports of research and evaluation findings are generally technical and not written for broad audiences, and that cripples our efforts to see to it that the full impact and benefit of research findings actually of any kind in fact take place. There’s a need to vary the delivery format required for such diverse groups as parents, school personnel, private sector, government, and program participants considering what is written, how it’s written, to whom it’s being presented and by whom it is being presented. There were other comments that dissemination findings are not even finding their way to universities and this is seen when we see that frequently research findings are not getting into or impacting teacher training programs, so that frequently feel that we know much more about how to effectively train teachers, prepare teachers, then we see taking place. The fact of inadequate dissemination has become increasingly important once again with the diverse population.

Other kinds of capacity building required. Networks organized and supported, funding of faculty exchanges, workshops on developing research skills and effective uses of research findings would be other things that would be important. Another interesting approach to building capacity is seen in the action research of Stafford Hood(?), he’s at the University of Illinois, and work that he is doing with teachers in empowering them to be full participants so that teachers can use the knowledge in both a formal and informal way and perhaps become part of the pool of those in education research and evaluation.

The fourth question has to do with the incentives and disincentives for traditionally under served populations. A big disincentive that we’ve heard over and over and over is that it is time consuming when many schools are struggling with the requirements of No Child Left Behind and some are facing being reconstituted right now and a variety of other things. The kinds of incentives that will be effective may vary and we have to get more creative about that but certainly one of the things has to be resources and money available.

And finally what we can expect to gain from randomized field trials in education. To me the most powerful and the most important and reason that I want to find ways in which we can use randomized approaches where appropriate is that systematic use of research is powerful to inform education policy.

Thank you.

[Applause.]