DR. ORLAND: Good morning. I am Marty Orland of the Center for Education at the National Academies, and it is a pleasure to welcome you all to this National Forum on Applying Multiple Social Science Methods to Educational Problems.

In keeping with the theme of the day I have multiple objectives for these brief opening remarks. First I want to familiarize you just a bit with the work of the National Academies and our Center for Education. Second, I want to discuss again very briefly why we view this forum as an important convening activity. Finally, I want to go over how we have structured the day and different sessions that comprise it.

The National Academies actually consist of four distinct organizations. Three are leading honorific societies in their fields, the National Academy of Sciences which began in 1863 under congressional charter, the National Academy of Engineering which was added to the Academies in 1964, and the Institute of Medicine which was added in 1970.

The National Research Council is, as its name implies, the operating research arm for all of the National Academies and in fact this activity operates under the auspices of the National Research Council and its Division of Behavioral and Social Sciences and Education.

We have at the Academies an interesting relationship in terms of the relationship to the federal government. As I mentioned, the National Academy of Sciences was chartered by the federal government in 1863, during the Lincoln administration. However, we are not an agency of the federal government, and that independence from the federal government allows us to provide the kind of independent, balanced and objective advice that the federal government in its wisdom indicates that it would benefit from. So, we think that we play a unique role in the panoply of organizations that advise the federal government. We do have a special relationship in terms of the charter and certain expectations for our performance. At the same time we jealously guard our independence and our commitment to objectivity and balance in everything we do.

The Center for Education, which I direct, is an entity as I said under the National Research Council and it has the following mission: (1) to improve education for all learners by promoting evidence-based decision making in education, policy and practice; (2) enhancing the capacity for educational improvement; and (3) focusing objective independent and interdisciplinary attention on education problems and the search for their solutions.

We take each of these obligations very seriously, and in fact this gathering is a reflection of the extent to which we do take it seriously. The Center operates under a core grant from the National Science Foundation as well as operating grants for particular projects from a number of entities both within government and in the foundation community and private sector.

How do we do our work? Three ways. One, we have four standing boards within the Center for Education. We have a board for our Center and we have three other boards devoted to specific substantive issues in education. We have a Board on Testing and Assessment, have a Board on Science Education, and we have a Mathematical Sciences Education Board. Each of these is intended to provide the kind of direction and continuity and support from the nation's leading experts on those domains and to provide the kind of guidance and support for the operations of the Center.

The operational work of the Center takes place through consensus studies, which is how the Academies in general does its business, as well as convening activities such as this special event. And particularly relevant to the discussion today, we are not a newcomer to this issue of quality of research in education and just recently in the last several years we have published a number of studies and in fact they are listed in your packet. If you wish to order any of these we have a 25 percent discount available.

Scientific Research in Education (SRE) is something that I think most of you are familiar with, published a couple of years back. This past year we have published four additional reports in the general topic area. We published a workshop report on implementing randomized field trials in education. We published a report on strengthening the peer review system in federal agencies, supporting educational research and just most recently; in fact, it is only in pre-publication copy but should be available I believe by the end of the month, Advancing Scientific Research In Education, which is a successor volume to the original SRE and last but not least this spring we published the volume Evaluating Curricular Effectiveness: Judging the Quality of K - 12 Mathematics Evaluations which we consider to be a very important and trailblazing report with respect to that area with implications more generally for the pursuit of scientific research in education.

As I said, this forum is really intended to be a convening function for the Center to bring together diverse points of view and interdisciplinary expertise to the topic of multiple methods applications to important educational problems.

The fact that three distinct organizations, American Psychological Association, American Educational Research Association and NSF have come together around this topic and that the interest in this event is so high as evidence by the full house we have here and the satellite room we have across the way is I believe testimony to the salience of this issue at this time.

One question that might be asked is why such interest, why now? I do think the interest reflects a number of key shared assumptions and I just want to briefly go over them and maybe this will provide some introductory fodder to what will come on later on.

The first assumption is that there is now an unprecedented interest in issues of methodological rigor and quality in educational research with particular attention to the use of randomized field trials or strong quasi-experimental research design to inform policy relevant causal questions.

We have heard at nauseam the reference to 111 references in the No Child Left Behind Act to “high quality, scientifically based research.” The Educational Sciences Reform Act also specified an agenda with regard to the increasing use of strong rigorous methods including and heavily emphasizing the use of randomized trials.

Even recently in the IES appropriations for those who have been following it there was specific reference for preferences to designs that would maximize the ability to conduct randomized experiments in education with the funding of future service programs and the relationship of the Institute of Education Sciences to other program units within the department to help foster that.

So, I think the fact that there is this attention to this with the federal leadership role is unquestioned.

Two, the second assumption is that I think there is widespread acceptance of the general proposition that judgments about the appropriateness of research methods cannot occur in the abstract but need to be placed in the context of their match with the questions that they are intended to inform. The notion of methods matching questions I think is something that has been a consistent theme including in our own volumes, Scientific Research in Education and Advancing Scientific Research in Education. So, that is the second assumption. Maybe we don't all share it, but I think most of us do.

Third, educational decisional makers often require answers to not just one but a number of critical questions in addressing issues of policy or practice and this suggests that research designs, programs and syntheses incorporating findings from multiple methods can be particularly powerful and relevant instrumentalities for meeting their needs. Again, it is an assumption that I think most of us share but part of the day is to find out how much we share these assumptions.

Fourth, despite this potential the application of multiple methods research approaches to educational problems is an underdeveloped area both conceptually and in practice.

It is really that fourth assumption, which I think logically follows from the first three, that was the genesis for the organizations coming together to say, "Can we have a convening event that can build a foundation around this notion of multiple methods research and what it means in education?"

So, we are going to explore this development during the day. We will begin with two morning sessions that attempt to provide conceptual foundations for multiple methods investigations. Hopefully most of you have had the opportunity to look at the paper that we commissioned for this event, “Learning from Attempts to Improve Schooling: The Contributions of Methodological Diversity” by Steve Raudenbush and that will be the focus of the first session.

The second session will be a discussion centered around the application of multiple methods approaches to domains outside of education. One of the contributions we think we can make here is to broaden the perspective of the extent to which it is not only educators and education researchers that are struggling with this issue; this is a consistent theme throughout the social sciences and in different policy areas.

We will then have two sessions to explore specific uses of multiple methods approaches in education. So, we will get from the conceptual to the more practice.

The first will be a breakout session in which we will have examples of multiple methods research investigations.

In your packet I believe you have brief abstracts of programs that we are going to be looking at and you have hopefully chosen the session that you want to attend, and we look forward to an interesting conversation in each of these breakouts around the designs and implications of multiple methods designs to inform those particular policy questions.

Then following lunch we are going to have a moderated conversation. This is really something that excited the Planning Committee. Excitement may not quite be the description. We thought if we could get an education decision maker and some researchers together, in something close to real time, to explore a particular policy issue or series of policy issues that are facing the decision maker and have a conversation about how a research program that would incorporate multiple research methods can inform those decisions then we could provide the kind of foundation and common understandings for the applicability, uses and challenges of multiple methods investigations.

So, we are going to try that and we will see how it works and that will be something which you should all stay for and give us your full review after the fact.

The next two afternoon sessions we are going to explore the implications of a multiple methods approach for two critical components of the education research infrastructure.

First we will look at the implications of multiple methods investigations for the training of educational researchers and then in the next session we will discuss implications for the funders of educational research.

Then we will conclude the day with some summary comments, and discussions of next steps. I should point out that we have a strong bias here against long presentations. There is a lot of talent in the audience as well as in the presenters and we have purposely designed each of our sessions to maximize opportunities for extended interactive discussions between panelists themselves as well as with the audience.

So, I look forward to a stimulating and instructive day for all. Before moving on I want to specifically thank our event co-sponsors and our planning group, from the American Psychological Association's Education and Science Directorates and maybe you could stand for just a second, Rena Subotnik and Merry Bullock, and what about Greg White and Heidi Sickler, are they here today? From the American Educational Research Association Felice Levine and Jerry Sroufe, from NSF Education and Human Resources Directorate Janice Earle and Barbara Olds and last but not least I want to thank my own staff, Dorothy Majewski and Tina Winters especially.

(Applause.)

DR. ORLAND: Two logistical announcements before we get started. The meeting is being audiotaped. There will be audio files as well as transcripts posted on our web site which we think is good but it does require or at least makes it more useful if we actually have those that are participating in the Q&A portion of this to come to the microphone, ask their questions and identify themselves prior to asking the question.

Second, as you can see we have had a tremendous response to this forum and as a result have not only a full house in this room but a satellite room, Room 110 which is just down the hall.

We don't want to disadvantage anyone in the overflow room from the opportunity to ask questions. So, what I would ask is that during the very end of the panel session when we go to questions and answers we are going to announce that we are going to move to questions and answers and for people in the overflow room who would like to ask questions if they would be good enough to come to Room 100 and we will do our best to accommodate them.

It is now my pleasure to introduce Rena Subotnik, Director for the Center for Psychology in the Schools and Education Program at the American Psychological Association to provide some opening remarks and then kick off the first session.

Thank you.