AGENDA ITEM: Session 5: Opportunities, Challenges,

and Lessons Learned

DR. HIXSON; Now, we have a panel of our Steering Committee, right, that is going to talk about opportunities, challenges and lessons learned, but we all want to wake up now for both Session 5 which is limited to opportunities, challenges and lessons learned in somewhat past tense and then Joan is going to be moderating at four-fifteen next steps for educational research positions in STEM disciplines and these are the two categories, the three-fifteen time and the four-fifteen time for the kinds of things that NSF would really like to get out of this workshop.

We would like to hear about where you think things stand with this kind of emphasis on what you think needs work and how and how you would like to see things move in the future because those are the kinds of suggestions that we are really after in terms of when we set programs or decide what kinds of things to fund. We are really interested in what the community has to say about what would help these kinds of positions if you think they should be helped. Now, maybe you have listened all day and say, "Ah, big mistake that these positions are under way."

So, I don't know, do the three of you have particular things you would like to say?

DR. FERRINI-MUNDY: I will just say a couple of things but I am assuming this will be a full group discussion. Through the day though one point that I guess I would make to start is that I am really very heartened and excited by this conversation because I think that what you are hearing especially from junior people is that there is really strong commitment out there across the country to making this work, to having fields of STEM discipline educational research based in disciplinary departments and having that grow and so on. So, that is really exciting and I think it presents some interesting opportunities. One is that we actually probably through the day have identified at least three if not more kind of different, I don't know if you would call them different flavors or different brands or possibly they are even different strands of work in this STEM education research.

One is sort of more traditional basic scholarly educational research work that is based in the disciplines. Another is more oriented toward the improvement of practice and specific studies and activities and development work that is intended to make a local difference and then yet a third may be the sort of service an outreach to the rest of the university and to the K-12 world that is part of this. So, an opportunity might be to imagine that it is a good moment for an organization like the NSF or the NRC through its kind of convening and framing of issues power to see what it would take to really bring coherence and strength and connections across these three areas.

The other thing is that this is obviously a growing field and organizations like the NSF and the NRC can do a lot to spark the growth and development of an area by starting to help frame the main research questions, the key issues of importance to be studied and so forth and it seems like there is an eagerness and readiness for that, and in particular this notion that there must be serious problems out there that need to be solved and how can we work on them and what kinds of preparations for doctoral students and what kinds of supports for faculty would be necessary? We have heard mostly about challenges it seems to me. What does it take to build the institutional infrastructure to support this kind of work; how do we evaluate faculty who are in this area; what kinds of expectations are reasonable? Where is the intellectual base and community in a university for this group particularly when we have this lone wolf syndrome of individuals in departments rather than a collective and then how do we get clearer about all these different variants? Again, we have heard today talk about the scholarship of teaching, talk about border-like scholarship. Maybe those are the same thing, evaluation assessment research, service, all of these variants of the activity that gets done in this field, how do those fit together? How do we cope with the strong service demands that everybody seems to be telling us about and how could we study the development of this field and the opportunities, the supply and demand issues?

There were also a lot of lessons today I thought. We heard that sometimes a structure inside of a university, a center, a department, an institute can help with the issues around evaluation and around resources because it is more centrally recognized than a single individual trying to kind of make a case.

Having a critical mass seems to be important. Taking a view of this field as truly interdisciplinary might be a way to tie it better to the experiences of scientists particularly and then scientists, engineers and mathematicians who are accustomed to thinking about interdisciplinary work and then I think there was also suggestion that endowed chairs, named professorships and so forth can bring credibility and sort of authoritative institutional endorsement of these areas when those are possible, so, a very rich discussion but no real clear solutions yet.

DR. SMITH: It looks like my name is next. I agree with Joan that it is really exciting especially when you hear that despite all the potential obstacles they are embracing this kind of work studying ourselves. I think it is really important and it is also encouraging to hear some of those senior folks and administrative folks paying attention, creating spaces and opportunities because that is another very important thing.

Maura reminded me to perhaps do a little more shameless self-promotion. Maybe it is shameful. I am going to pass this around which describes the annals or research on engineering education. This was Norman Fortenberry's idea. He was at NSF for a long time and now he is here, actually here in this building. He is the Director of the Center for the Advancement of Scholarship on Engineering Education at the National Academy of Engineering.

This is quite a change for the National Academy of Engineering to start up an organization like this. One of CASEE's projects funded by the digital STEM library is called annals of research on engineering education and since this is an emerging discipline what Norman is interested in and what he recruited me to help with was how do we reach consensus about high quality engineering education research and how do we bring people into this conversation and help them learn how to do engineering education research and so this is a secondary journal. Actually it is a web portal. We now have 11 journals and we included more though in engineering. There was a biology journal. There is a computer science education journal. Our associate editors recommend articles that are involved in educational research in the STEM disciplines and the authors accept the invitation. They write an extended abstract because they may not have access and then they respond with reflective essays, the kind of reflection that has been occurring throughout the day. What does this work mean? How do we come up with a researchable question? That emerged several times today. Now, how do people frame researchable questions and I am reminded I was just reading I think it is Getzels’paper, the problem of the problem, you know how difficult it is to come up with researchable questions and so that is one of the things we are trying to do. It is an adventure. I would urge you to visit it and consider it contributing to the comments but we still haven't been overwhelmed with participation. Michael was commenting on the activity on that. It sounded like a physics ed research listserve.

DR. ZOLLMAN; There is a physics ed research and then is a young faculty, a young person, early career.

DR. SMITH; We are hoping that there will be similar activity on this and I think that is something else that as a community or as communities we may want to consider, how do we connect these folks who by and large are off by themselves in more traditional departments and idealogues or what is called blogs or all these other things may be a way to continue this kind of conversation.

Some other things that are happening that you may want to look into and you may be able to help us especially in areas that are when you are doing this kind of work longer. There are 17 or 18 centers for learning and teaching, NSF-funded centers?

DR. HIXSON: Something like that.

DR. SMITH: Most of them are K-12. There are three higher ed centers, one at Wisconsin, Michigan State and Penn State called the Center for Integration Research Teaching and Learning looking at STEM PhDs and the preparation of STEM PhDs and one of the statistics, I am on the advisory board, they indicated that there are about 100 research intensive universities who graduate 80 percent of the STEM PhDs. Only about half of them go into academe. Only 1 in 20 goes to an R1 institutions, a research extensive university. Some folks would argue that most of them are being prepared to go into research extensive universities. Is that really the kind of preparation that we want? It is an interesting question but it has some ties with our conversation today.

Then there is the Center for Advancing of Engineering Education and I think Louis mentioned this earlier today, that is at the University of Washington and the other is a research center doing research on engineering education. So, that is pretty interesting.

The last thing I will mention is how do you build this community which is probably the main thing I am interested in right now is we want to do more systematic research in engineering education. It takes knowledgeable skillful people to do it. How do we develop capabilities? Physics I think pretty much has done this. Computer science hasn't yet. Biology is working on it. Geoscience is considering it. So, how do we build it? CCLI funded a little project to help us with this.

PARTICIPANT: Susan, too.

DR. SMITH: Susan, too. Oh, thank you, Susan. You are the one who gave us the hardest time. It is a little project called Rigorous Research in Engineering Education Cultivating a Community of Practice and there are 20 engineering faculty per year. They will be learning how to deal with engineering education research more so from folks from AERA Division I. If that is a group you are not familiar with and you are interested in STEM education research I suggest you take a look at American Educational Research Association, Division I. Maybe that is not the most relevant one for many of you. It is called Education of Professionals. It is perfect for engineering but it is mostly medical school folks now, but they are really open to having folks from other disciplines. They really know how to do this kind of work and so we have had some great partnerships with them especially in the workshop when they do a long project.

The first year a few years ago the funding came kind of late and so I sent them the engineering dean's list just a few days before Memorial Day for a July workshop. My colleague, Ruth Streveler went off to Kenya away from any kind of communication access and we thought we would have to call people and cajole them to send their colleagues to this. Within 24 hours after the message was sent we had 17 applicants for 20 slots. Five days later we had almost 90 applicants for 20 slots, and they had to be engineering faculty. We were stunned. In the second year we raised the expectations a lot. Still we had 50 applicants and so there is interest out there and it is not only really for faculty although it is really for folks willing to run the risks of doing what you love to do, but there are others and I think one of the things that sessions like this can help with is to make the connections among folks who are interested in this kind of work and it is really neat that it is not just engineering. It is physics, chemistry and biology, geosciences and computer sciences. So, keep connecting. Now, that was a lot of shameful commercial.

(Laughter.)

DR. HIXSON: Although those are excellent for the other disciplines. So, are the other disciplines are listening because you, too, could propose exactly the same things and it is innovative.

DR. ZOLLMAN; Let me follow up a little bit on lessons learned that Joan mentioned. In fact she mentioned things like endowed chairs. I would like to add to that awards outside your department and particularly awards outside of teaching awards and make a plug to NSF that I think the demise of the distinguished teaching scholar is really a blow to this sort of thing, that there was a program where this is not going to affect the early career people directly but the fact that NSF was recognizing this in a broader sense than just within education and human resources with the end of that program or the hiatus of that program people are looking at it differently I suppose is one way to say it and that goes broader, too, in the sense that the disciplines need to be looking more at what is possible to recognize individuals within the discipline and this, I mean like AAPT has several awards. In some ways those awards are not quite as noticed by my colleagues as say fellowship in the American Physical Society which is also possible for people in education, but it has only been possible in recent years and so I don't know who exactly encourages those things but it is something that needs to be done for just general recognition of the discipline and in the meantime for the younger faculty continuing to make sure they get career awards is also very valuable because that type of award and being part of the PECASE, what does PECASE stand for, presidential, early career something something is also a very valuable thing to have folks within education and human resources within NSF being nominated for those awards is very valuable.

Most all the things have been said but let me just bring up one other thing that we talked about a little bit and in fact I asked the question specifically and toward the end of our last group it was focus on what is positive about the opportunities that one has for doing this kind of research in disciplines and people came up with a list that was at least as good as one that I would come up with by myself and obviously I must think there is some opportunity there or I wouldn't have stuck with it for so damn long but let me just mention a few just that again can build on both in terms of what NSF might be interested in but the rest of us are as well and I will do this rather quickly without attributing it to individuals but just so I can get through it quickly.

Certainly using our own classrooms as our laboratory and using our colleagues' classrooms which both means you have got things you can do within the framework of your teaching assignments or other teaching assignments that really integrate your teaching and research in a way that most of our other colleagues cannot do is something that really drives a lot of work we are doing and then sometimes I can see a difference in the way students learn is even better and that in itself sometimes leads to credibility with our colleagues and so do these other things, peer review and so forth about issues relating both to teaching and about educational research. Our colleagues raised questions because we are able to interact with folks that are doing research and whether they are trying to explain that to their mother-in-law or students or anyone else you know it leads to questions of how do I deal with this and in fact some of those questions led to a major career move for me where I was able to and still am working on issues related to teaching quantum mechanics and so these kinds of questions wouldn't come if we were not in the discipline and then finally a comment that Joe made in the earlier session as made in here is still very important that the perspective we bring because we know the discipline well is very important in trying to understand why students are or aren't learning that discipline and so we can't get that unless we really know that discipline very well.

So, all of these things kind of go into a package that makes this an incredibly valuable and useful way to spend a career in spite of some of the other things we have talked about today.

DR. HIXSON: Great observations and now do other people have questions you would like to ask or comments that you would like to make about opportunities, challenges and lessons learned especially things you want to be sure are on the record here?

DR. RASMUSSEN: One of the challenges that physics education and to some extent math education have figured out some things and to what extent are those lessons useful for newer areas like geology or you know is there a sense that they have to reinvent the wheel themselves? I mean perhaps there is a part of that that is true but are there aspects of what physics and math education have figured out that could be useful lessons for the other disciplines? It is a very good challenge and opportunity both.

DR. ZOLLMAN; That actually came up. That sort of issue came up in my mind during some of our telephone conversations and conferences when we were discussing organizing this and it occurred to me that at least within physics education we have not really studied ourselves at this point and I mean we have all sorts of anecdotal information available. I was part of a proposal with one other university to try to do a study. That proposal was not funded and this is not a plug for that proposal because it had some real problems. The other university involved was the University of Colorado where a Nobel Laureate was leading the charge in physics education. So, it doesn't make it very typical and the reviewers picked up on that but in any case we really have not at this point studied ourselves in the sense that what have we learned over the last 30 years that would be useful even for our future and maybe it is time to do that and see if there are lessons that will apply. I think at least from a few of you I have learned some things that probably wouldn't apply in the same way that my career went and in fact wouldn't apply even for a new person coming into physics education. My career started when the discipline was quite immature and it is quite different today but that is a good point and I don't know that we know a lot.

DR. FERRINI-MUNDY: I would point out a resource. NCTM produced a couple of years ago a two-volume set on the history of mathematics education for the last 50 years, and so it is not only about mathematics education research. It spans a range of topics, but there is a chapter about the role of federal agencies in supporting the growth of the field and there is a chapter on research and so on, but I would agree, I am not so sure that there has been direct study of the development of the field. These are commentaries provided retrospectively.

DR. HIXSON: One historical thing and just think about this, but one thing that showed up in our education is the engineering education coalitions began what 15 or more years ago, and so there was a lot of money put in at single times in a few years to large grants across the country for groups of people in engineering to basically, there were several things they were supposed to do but fundamentally curricular work. I don't think in any sense this was supposed to be research in teaching and learning. A lot of money went in and then if we track now in our division people can trace the ancestry of many of the people that are doing research now back. Then the next thing that happened this past summer there was a chemistry group conference where for the first time there was a whole series of talks during the week and they moved pretty much into what could pass for chemistry education research as opposed to what I did in my classroom without any analysis about what happened, and at that time people were pointing out that many of these projects could track their origins often through grandparents and so forth back to the chemistry initiative which was again a curriculum focused event which was $10 million that started about a decade ago and went until about 5 or 6 years ago. So, a slug of money went in across the nation to do curriculum reform and what came out of that for a whole lot of explainable reasons when you look back. So, one of the things we are kind of wondering is whether, now, you could argue all day as to whether the engineering coalitions really did everything they were meant to do about curriculum development. You could argue the same thing in chemistry but what I think we can sort of argue is that they did lead to these fields. So, it may be that the reason biology and geology are having trouble is because nobody has put the slug of money in that is called fixture curriculum which allows everybody to participate. You don't have to be an expert in anything. So, any kind of faculty can come in the beginning but then the spinoff becomes the development of this field.

So, it is just something to think about whether if someone wants to do a study looking backwards whether that has been some of the reason or it just happened.

DR. PARSON: But what is interesting is that in biology, and I think you could attest to this many of the people in biology that are doing this sort of thing came out of programs where you didn't know that was biology. It was like CTP, collaborative teacher preparation which went across all disciplines or GK12 that goes across all disciplines.

So what happened within a specific discipline is not as apparent as it is when you have what they have within physics when they really put bit slugs of money into what was going on in physics, big sums of money into chemistry. So, I think you are right about that but I think, too, if you look and you put big slugs of money in you begin to get people who sort of get brought into it. They get caught up in it and develop and people who tell me they never would have done what they are doing now if it were not for those programs. So, I think you are right, but I think we have to expand it a little to look at those which were big programs that were not so discipline specific to get big slugs of money so you can identify a project.

DR. SMITH: Since Maura asked me to do some shameless promotion I will ask her to do some shameless promotion. We are studying, Maura is studying the participants in the rigors of the engineering education project which is R20 from the CCLI plus another 30 that are funded through HBCUs. So, there are 50 of them here and Maura has followed up and did you ask the question about the influence of the coalition? That would be really a good question. How many of those folks participated like PK in one of the coalition projects and then got interested in engineering education research?

DR. BORREGO: We have follow-up plans but totally separate from that I have a coalition project going where I have gone back and interviewed some of the people that were involved and look at the web sites and I was focusing on the dissemination but it is one of these projects that is start out a little bit because of the need to figure out exactly what is the framework and where am I going with it but I already have some data to analyze it. We are finding some really neat things about how someone can come from an engineering background and what are the problems that they have when they start to learn education research; you know, what are the two or three concepts that they are really bumping into and running over and also attitude changes about being open to help.

DR. PARSON: Are you looking at the origins of the --

DR. BORREGO: Yes, and it might be that engineering people has said that what will come out of these is not easily identifiable engineering stuff but yet they were the source and it would be interesting for us to know because it tells us things we cannot measure within those programs but yet are important to know. They are the unexpected outcomes. Do you know what I mean? There are interesting things that you can find as a result of the coalition but when you go in to assess the coalition it is not what you --

DR. PARSON: Right, but if you look to see where are the people now, where do they first get drawn in.

DR. TANNER: Causality is always a problem and there has been no lack of money in terms of biology education. There has been a lot of biology curriculum development. That is sort of a question. I can't sort of deal with geoscience but what really is different then about the magnitude of money that has been put into chemistry versus biology? Who has been awarded those grants? What directors did they come under or how they were managed because I don't think that biology has been lacking in funds. I mean I can think of lots of personal development projects that have not led to an increase in the number of education researchers.

DR.HIXSON: It is critical mass simultaneously, in other words each, it is true of the engineering coalition, true of the chemistry ones. They have to be in large groups of people working together and all within the same few years. So, you are not talking about different smaller projects.

So, you are talking about a $2 million grant, five of them operating within 3 years.

DR. TANNER; So, anyway I just asked a question about biology and then to add onto that I feel like the other pieces when I think about projects like that and who would be PI on that what would look like and it is interesting to figure out how this has somehow led in chemistry. I would like to know how it has led to more young education researchers in the field because those grants are by and large going to go to pretty senior faculty that are already established.

DR. LEWIS: They get postdocs. That is why I was able to do this because I could spend 2 years learning how without the need to have a different kind of a postdoc.

DR. TANNER; I don't disagree but I think there are other postdoc programs. So, I am trying to bring the focus on you don't necessarily need a giant curriculum grant. You need training programs. You need postdoctoral programs and maybe they are in the context of curriculum and maybe they are not or you need --

DR. LEWIS: I think it is just something that creates a community of faculty talking about educational improvements. Without a huge grant there is perhaps not the impetus for many faculty to have that conversation.

DR. TANNER; I am just going to continue to push it. I mean there have not been a lot of biology folks who sit around and have lots of conversations about curriculum development in biology.

DR. HIXSON: I am not saying it is the only way. I am saying it is funny because when you go back and evaluate is it always and so did the chemistry curriculum change or can you point to this and this and so that is where we always were and all of a sudden at this conference people were counting the number of people and said, "Look, maybe we don't," so it is not to say that that is the only way to do it and it certainly might not be the best design but it is kind of an interesting thing that we are noticing now.

DR. PARSON: Because I have to leave I am going to sort of push in. This is sort of specifically biology which is to let you know we have been talking with HHMI. We have been talking with NIH to say, "How do we get the biology community together?" because one of the real problems that perhaps is true within that community which is not particularly true in the others is there is no one place for the biology education people to get together and talk to each other across all these subdisciplines.

DR. TANNER; And the only places there are in the K-12 sector and if you look at all the people who are young education researchers if there is one connection it is that they were all involved in a funded K-12 outreach program through SEP or through HHMI pre-college or HHMI undergraduate work.

DR. LIBARKIN: I wouldn't say it has been a decade but there is a program in the geosciences with faculty development, you know, appearing in faculty sort of kinds of things and talking about visualization and energy of sciences and that sort of thing and I can actually say that I was involved in one of those workshops for new faculty and I just got an e-mail from a participant in that faculty who she and I had had conversation about designing a research project, and she took those ideas and wrote a proposal and got funded and she thanked me for just pushing her. I think that conversation would never have happened without that huge chunk of money. That is what you are talking about, but the question is in a decade will that be sustained. I will be curious to see.

DR. HEPPERT: I want to know whether we have moved on, whether we have sort of made the transition on into future opportunities yet. No? Okay.

DR. ZOLLMAN; I just wanted to make a point that talking about training grants and things like that you guys have, some of you have used very effectively but there is also the individual grant. I never put in a proposal without a postdoc and a graduate student. That makes it a little expensive from the NSF viewpoint but for the most part they have not complained. Well, they have not turned me down for that reason but it is also very important that educational research and development grants in the disciplines need to have postdoctoral people in them and those postdocs may come from people who get degrees in disciplinary education but they may also come from people who are looking to move into that field after they have gotten their PhD and so that is something that all of us need to be looking at independent of age that when we are putting in proposals are we in fact making sure that we are doing what we need to do to help train the next generation by using some of that funding both to get the research done at a relatively cheap rate and at the same time train future generations of faculty.

DR. SMITH: I agree that we are in the talent development business.

DR. ZOLLMAN; Yes, and we are. We are academic institutions. We ought to be in that particular field.

DR. WOODIN: I just want to put a word of caution in there. I think how programs respond to budgets like that will vary among disciplines and some disciplines are not as generous and it is really important to justify the role of the graduate student or postdoc in the program because they have to be part of the project, an integral part of the project and you have to be real careful about that.

DR. ZOLLMAN; I am glad you said that physics was generous after Duncan left.

(Laughter.)

DR. LEWIS: One of the conversations that we have had in this discussion was about the role of evaluation in terms of these fields and that many of us are now doing project evaluation, and it seems to me that that has been a double-edged sword because that means that neither the PI of the project nor the evaluator is really directing their own research program and neither one has the kind of money for a full postdoc for example because it has to be split and my impression is that it is more the direction that things are going rather than less and I guess it worries me a little bit. I am perfectly willing to keep trying to do research in that way but it seems definitely that is it a cobbled-together project always because of the evaluation emphasis which is different from a basic research emphasis.

DR. PIMMEL: As we have been talking in our group what you have to do is make the evaluation one of the items that is on your research agenda and begin doing research on how to do evaluations and you have got all these projects and you can try different approaches.

DR. LEWIS: I would argue that there are many social scientists who are much better at that work than I. So, I don't know the special qualifications that I bring as a chemical educator.

DR. WOODIN: You are connected to a discipline.

DR. LEWIS; The PIs are also connected to the discipline. It is just I am not sure how to deal with that.

DR. ZOLLMAN; There are some folks that I have worked with who use the phrase "research-based evaluation," and so they try to move forward their research agenda at the same time they are doing the evaluation and actually the group I am thinking of has been headed by Owen Seymour at Colorado. So, there are some real name brand people who do that kind of work.

DR. HIXSON: So, what you need us to do is you need to tell NSF that you need X thousands of dollars to support a postdoc in evaluation and then we just go through our list and when we get Jennifer up to her X thousand we move on but I am semi-serious or I am very serious that we do get a lot of inquiries. So, if there were some external way of hooking up, in other words that could be a mean of support for a postdoc who would be doing the evaluation. So, it wouldn't take at least your time away.

DR. LEWIS: These sorts of money are not enough.

DR.HIXSON: But that is what I mean. So, you might need a couple of different projects at a given time.

DR.LEWIS: More like 10.

DR. HIXSON: Or 10.

DR. COOPER: I think what Jennifer is getting at and this came up at the Goin(?) conference as well was Jennifer is the evaluator on our projects and I think the way Stacy put it was that she was in chemical education research and why should she have to have an evaluator in her project; what was the question and because you can see why a curriculum development project would need an evaluator but if you are doing the research and if I were doing synthesizing new heterocycles for example I wouldn't have an evaluator on the project. I would publish my results and it would be peer reviewed so that the question is will we ever move to a situation where if you are doing an education project within a discipline that you won't need to have an external evaluator and I guess that was the question.

DR. HIXSON: Actually I think the direction things are going is that if you are doing synthesis of heterocycles I think the day will come at NSF where you will be asked to have an evaluator for that project. Do you understand what this discussion is all about? In other words the person who stands back and says, "Are you really going where you say you are going; are you using your resources well?"

DR. COOPER: You gave a good answer to that. I think that is what Jennifer might have been getting at.

DR. LEWIS: It is tough. The extent to which that role is are you doing what you said you were going to do, that is not research. So, if that is where chemical educators are going to get some funding to do research that can't be the rule. That has to be something else.

DR. FERRINI-MUNDY: My worry about that direction though is that it is already a thin field. I mean the folks around this table are central and it is not as though there are hundreds more of you out there and if the main avenues toward funding are sort of by attaching to somebody else's grant or being a helper on the education piece of a scientist's research grant or by doing evaluations and so forth what I think falls to the wayside there is the more coordinated coherent work needed to solve the problems in the field or to address. Solving is not the right word but to address the serious problems in the field.

So, if there are serious problems about chemistry teaching and learning that need basic research and that need discipline-based expertise to address, you know you do worry about the overall coherence of the field if the funding doesn't line up well with the problems.

DR. TANNER; The other thing with needing to go back to the issues is that as I was sitting here and I was talking to Michael and I said something about it he said, "Oh, we wouldn't advertise there. We wouldn't go there. You know, I would go to a couple of very specific places," that I feel like discipline-based education researchers are also very, biology is fragmented but then education researchers are fragmented as a group and cloistered in a discipline appropriately but what it means is that as a biologist educator I have a hard time accessing the expertise enough to learn in math education or physics education and so I think that NSF and the National Academy could play a great role in continuing to do things like this where people are actually able to present their work and exchange information on methodological issues.

I mean Julie and I keep trying to catch time in breaks and we have yet to actually be able to talk and so I think there is a real need to kind of foster that community across disciplines as well and I was glad to see this piece that Karl handed out. I was wondering like is Cell Biology Education on there and what journals are on there. So, I think it is great but we just seem to be kind of the discipline-specific aspect has also cloistered us in what is somewhat an unhelpful way.

DR. IMBRIE: I have repeatedly gone out to try to find NSF reports on stuff that people have done. I go out and find an abstract and I want to know exactly what they found and you go try and Google Scholar helps me because you can actually go Google this person, the PI's name and go see what publications are out. It would be nice to have access to the final report. I mean it is all public domain because the government is paying for it all but that is not so easy to do or at least I have not found a facilitative way to be able to go through and pull out what other people have submitted to you even if it is simply the articles they have written so that you can go directly and find those articles associated with the different projects.

DR. HIXSON: That is on the agenda for NSF because at some point all the reports are supposed to be public. Our division does pull what people send in over into our project information resource system but again that requires that you come to our division and look at PIRS and it is also incredibly dependent on what people decide to put into their annual reports and final reports but long term that will probably help. Short term you are right. It is a problem.

DR. HEPPERT: I think in my mind we have in one sense crossed the line here. So, I am going to bring up an idea that has been kind of simmering for a while.

As you can tell I think there has been the sense that there are lots of lessons learned among the communities that are not necessarily completely shared with the other communities, the other education research communities within disciplines here. There was a sense I think expressed, too, that there is the opportunity through this dialogue to start to ask some critical questions about what might be the next big issues, big questions to ask, the big questions in terms of you know you can think about applications of technology in education, what are the next big issues we need to, as researchers we need to address; what are the next big research issues we need to address in terms of pedagogy or other kinds of issues. There is I would say a lot of preparative work going on right now for the future in the research disciplines, offering up as an example what James Duderstadt has done with the National Academy of Engineering in their report on meeting the challenges of a global economy and the idea that the US has to start to position itself technologically to compete in a future that is much more interdisciplinary, much more competitive, much more real time oriented. It strikes me that there is an opportunity to apply exactly that type of paradigm, broad interdisciplinary paradigm to educational research as well. I am not saying that there is the opportunity to do that in terms of existing NSF programs but if you think about the kinds of structures like the discovery institutes that he has talked about I am passionate enough about this that I think there ought to be a discovery institute focused on disciplinary education in science and math technology, engineering. There ought to be that kind of resource that cuts you know, I think their sense is that cuts across major themes in engineering. This is a theme that perhaps cuts across not only engineering but major themes in the sciences and technology and mathematics and engineering as well. That kind of over-arching structure could help foster interdisciplinary communication I think. It could raise, it could help raise issues of what are some of the bigger questions and probably in both an intentional and as Terry was pointing out a non-intentional and you were pointing out, too, Susan, in both an intentional and a non-intentional fashion. That kind of effort could really help to see the future of STEM education in a variety of different disciplines and in a way that is much more interactive than we have had in the past if that is the culture that young faculty tend to grow up in.

So, I would like to see us get more aggressive about seeking a place at the table in research with the kinds of efforts that the engineers are beginning to envision for the future. You know, they sort of call it engineering research centers on steroids as the sort of catch phrase.

DR.HIXSON: The National Academy today sort of conflicted with this has a, is it their committee meeting?

DR. HILTON: It is the Board on Science Education and they are meeting out in Irvine while we are meeting here and they are probably going to approve a new consensus study on discipline-specific STEM education research.

DR. HIXSON: And it is trying to look at what the status is within each of those fields.

DR. HILTON: Right and then certainly focus on the institutional and the researchers and that study would also address the research fields themselves, but Carl Wieman is the Chairman of that Board and they just need to officially approve it but I think they have probably approved it by now. I don't know. What time is it in California?

DR. HIXSON: This doesn't directly at all talk to what you were saying, Joe, I understand that, but then if they did this study those results would be published. So, that would be of some use to the disparate communities here.

DR. HEPPERT: Oh, sure.

DR. HIXSON: But it doesn't meet what you were talking about.

DR. HEPPERT: No, not at all and I am not obviously, I don't want to broaden this too much but in the current economic environment I am thinking about this would be perfect for priming the pump for the future and I think as these kinds of discussions happen and they are going to continue to happen, as people worry about our technological competitiveness we need to make the point that the foundation for that technological competitiveness is our educational competitiveness and the quality of our educational system.

DR. SMITH: It comes back to Joan's great idea.

DR. ZOLLMAN; Let me follow up just a little bit. There was a series of focus groups that met in this building about 2, 3 months ago. There was a request from Congress to write this report on how we are going to keep the United States competitive in the 21st century and as I understand it the report --

PARTICIPANT: The Gathering Storm?

DR. ZOLLMAN; Yes and I was on the education focus group and speaking of being a lone wolf I was a voice maybe in the wilderness. I don't know. Most of the people there thought the way to improve science education in this country was to offer more scholarships to high-end students and that is recommendation 2 or 3 in the report as I recall.

DR. FERRINI-MUNDY: I was a quieter lone wolf in the same room. It became clear that to try to bring research into the discussion was going to be impossible.

DR. ZOLLMAN; I gave up on the research. I kept saying, "You will not get students to become scientists and engineers if you don't change the way it is taught," and nobody listened.

PARTICIPANT: This whole report is online.

DR. ZOLLMAN; Yes, it is online, but what I am trying to do is not try to tell the story so much as to reinforce what Joe was saying that we need to become more aggressive in looking at our own future and making recommendations about our own future because nobody else is going to do it for us.

DR. HEPPERT: Thinking about, I won't say, clever ways but thinking about appropriate ways to tie this to issues that are of obvious currency to Congress and to the scientific community as well and global competitiveness, whether our educational programs from K through the grave are truly globally competitive or not and will be in the future is I think a critical issue that we should play off of.

DR. FERRINI-MUNDY: It is easy to lose sight of what the purpose of the work is when we are caught up in the institutional arrangements and in individual challenges, also.

Dean and I want to clarify we were actually in separate groups. He was in the higher ed discussion and I was in the K-12 discussion because I do remember I said a couple of things but he wouldn't have heard them because he was in a different room but the situation was similar.

DR. ZOLLMAN; The point is we both have had the same experience.

DR. SMITH; One other reaction if I may to Joe's comment about things to emphasize, earlier in the conversation about science education with a capital "S," or a capitol "E" it appears what we are emerging toward in this group is a capital "R" focusing on the research and P.K. reminded me of another thing, serious conversations called the engineering education research colloquies and hopefully P.K. will say more.

DR. IMBRIE: We are from the engineering perspective trying to look at what research is needed in engineering education in order to prepare our students for the future and so we have gone now through a series of two colloquies where we have used our ABET outcomes to say what is it that we want our students to be able to do; what research do we need to have in place in order to be able to show that they are doing what it is that they are supposed to be able to do and it started out we had 25 or 32 really core people go through the first one and map out research that needs to be done and then we broaden that to a group of 60 people in that second conversation where we have innovated on what the research themes are and we are slowly merging those down to research areas with research topics within each one of those. So, it is kind of doing what Dean had talked about. We have critically looked at ourselves and said, "Where are we lacking the information?" We have identified things in sciences, in mathematics, biology as here are things that we know we have to have or what industry says they want our students to be able to do and are trying to march towards what research is needed to be able to show that they are actually getting there.

DR. TANNER: That is your institution or your --

DR. IMBRIE: That is the engineering initiative.

DR. SMITH: It is an initiative beyond the engineering education community. It involved a whole lot of learning scientists you know like Jim Pellegrino is a part of. Oh, gosh, there is a whole bunch of folks who are really educational researchers who have been working with this.

DR.IMBRIE: So, we have tried to bring from mathematics education, chemistry education, physics education as well as the learning science community.

DR.TANNER; That sounds like a wonderful idea if there were a critical mass of biologists and people in science education research and I am sure other folks around the table, I mean even K-12 people around the table who engaged in that kind of research agenda setting and some traction on common methodologies would be so helpful to the field of biology education research.

DR. IMBRIE: From our perspective as we start to create our new, well, as we created our department we started going what are the research areas for engineering education, if it is going to blossom as a discipline what does it really need to focus on and that is really where the thrust of this has all come in trying to keep the -- the big part is trying to keep a large amount of -- Russ is my guardian of this. He keeps making sure we have the whole community involved or a non-trivial number of people from the community involved and that has been a big challenge. What processes do you set up so that you engage enough people to have enough input but not be so disparate that you don't ever get anything done and that has been a real challenge to come up with a process for being able to do that, in other words in part how do you expand beyond the choir.

DR. TANNER: When I talk to senior folks in biology about issues like those people who have done I think more curricular reform who talk about oh, God, I don't want to be in a group again to talk about curriculum issues but I think they would be very interested in talking about research issues and developing appropriate assessments.

DR. SMITH: I used to make the comment about faculty not wanting to hear about education research or engineering education research in light of you know it is like Woody Allen saying, "I am not afraid to die. I just don't want to be there when it happens." I am not afraid of education research. I just don't want to be there when it is talked about. I don't say that anymore. It really has changed.

DR. HIXSON: I think now we should give Joan a chance to do Session 6.