AGENDA ITEM: Session 3: Break-Out Discussions of Motivations:

Moderator Reflections

DR. ZOLLMAN; I have been asked to call everyone to order. Am I the only one who teaches large classes?

Okay, So, Karl volunteered to go first. We are going to reflect on what we heard and that for all of us I think is not going to be terribly easy. Well, it is either going to be very easy or very hard. So, Karl says that it is easy. We will let him go first.

DR.SMITH; Most of the folks in Room 1024 already had had a chance to tell their stories in response to those three questions and so the group had a few ideas and decided to pick on the NSF folks saying, "Where is the money going to come from?" and so, we heard from most, I think all of our NSF colleagues about changing funding opportunities within NSF and some ideas to keep the grant writing workshop which is about a 1-day workshop.

There was a little bit of conversation about the struggles between improving practice and the expectations for practice which we heard earlier today and your basic STEM education research. You know, it is something I grapple with regularly this connection between research and practice. Should it be tighter? Should it really focus more on research? Engineering is a profession and practice makes sense unless they are really floating a body of knowledge and Michael mentioned this that is not tied to some underlying theoretical framework it is not research and I think there is emerging agreement in that area and it sounded like from the conversation from our NSF colleagues that it is more support for building knowledge focusing on basic research and so anything else? Margaret, you were listening so attentively. Other folks from 1024?

DR. BOYLAN: I think just one other observation and that is that since I have been around for 20 years at NSF in the early part of DUE's life there was just so much powerful effect that could be had from funding people to implement better practices and teaching and we weren't worried about finding the global optimum. I mean there was just a cavernous, a huge opportunity for improvement that as time has ground down people are beginning to ask the question when are we going to be able to declare victory in the teaching wars here and what kind of systematic knowledge are we gaining from the billion dollars plus that we have poured into this and so I think if you look at it as a time dimension kind of issue you realize that eventually we do have to be more systematic and careful at some point about really works and under what circumstances and what kinds of students and what kinds of institutions and so on, what kinds of disciplines and that is why I think that the transition that we have been making in our CCLI program and other programs reflects this kind of challenge.

DR. ZOLLMAN; Okay, why don't I follow up on that because in fact we had some of this same discussion and maybe a little bit more in the framework of what are the expectations for faculty in this area. Is it to make local improvements in teaching or to do scholarly research or some combination of both? I don't think that we came to any conclusion that there is one size that fits even a vast majority of us but certainly the idea that someone who is appointed in a disciplinary department to do educational research will be expected to do scholarly research. That scholarly research may have lots of applications whether it is applicable locally or on a broader context or hopefully both. That is something that would be naturally part of the expectations for tenure and promotion and just for being a good faculty member as well.

I don't think we came up with anything totally unique in that but did kind of talk around the issues quite a bit of that kind of thing. We talked a little bit about why folks might think that it is appropriate to have this kind of research in a disciplinary department and I think Joe mentioned it this morning and then mentioned it again this afternoon and I think there is general agreement that there are things that people who are sociologists or psychologists or educators just simply will not be able to understand about our disciplines that we at least think we understand better than anybody else and that is an important part of doing educational research.

Whether or not those ideas are ones that will allow us to expand into more science disciplinary departments in the near future probably would have more to do with other issues like budgets and other priorities within hiring in departments than any one person or any group of people can control.

We had also talked a little bit about support. Duncan brought up the issue, I don't know if he worded it quite this way. Maybe I am actually putting words in his mouth of as the discipline-based physics education or disciplined-based education research expands, that is as those of us who are putting out graduate students put out more graduate students will there be enough money to go around, and I have worried about that for years. I am creating people who then compete with me for grant money and they have been quite successful.

(Laughter.)

DR. ZOLLMAN; And we know that the pot is not growing at the same rate that at least the physics education research community is growing. What does that mean both for us old folks who have become accustomed to having grant money and more importantly for the young folks for whom it is important to get some grant money in order to look good within their own local environment and in the end the ultimate peer review is whether or not you have got some money to spend on your own research and this I think is an issue that we just have to think about. We don't have a solution to that. We can all try to get Congress to put more money into the NSF budget but of course they can go in the other direction, at least for the NSF education budget. So, that is a major issue that I think is one that is going to affect the community and was one that we discussed for quite a bit of time.

DR. GOULD: That is not just physics education research. We have that problem in every field of physics.

DR. ZOLLMAN; Oh, yes. Right now at least given what the Administration proposed to Congress it was particularly emphasized, but as Kimberly and I were talking earlier today, NIH doesn't have the equivalent of these folks and that hurts all of us because the biology people would have to get money from the same source that the rest of us where I don't have to worry. Most of my biology colleagues who are doing other types of biology research really wouldn't look at NSF because they don't have any money and NIH has lots, but there you are. You are right. There is an issue here that science in general is not funding, is not being funded as well as it should be and as a result things are going down. I think pretty much covers it. That is probably enough.

DR. FERRINI-MUNDY: Our group also started off in a very polite way but I think with probably some underlying deeper issues, curious about why the NSF was interested in this topic and we were fortunate to have NSF folks in the room who could kind of speak to the evolution of some of the work in DOE and how it has gotten to a point where there seems to be a stronger need for evaluation and for understanding how things work and why things work and it was my sense that that sounded, at least to me it sounded reassuring that that is a track that is kind of in place the conversation then slipped over into well, what about the rest of NSF, you know, how does what might be going on in EHR actually intersect with the research directorates and how might it be the case that even at the NSF level the kind of thing that we are now hearing about is happening in STEM departments? Is there a version of that that happens across the NSF and where might things be headed, and we started down that road a little bit and then the group had a pretty wide-ranging conversation I think much of which I would characterize as a set of, I don't know, tensions or juxtapositions of kinds of issues that the people in the group face. So, one phrase that stuck with me was this notion of are we involved in doing educational research or are we seen as educational resources in our departments and I think that that revealed for some of the folks in the group this sense that what many of them may want to be doing is basic research, trying to better understand teaching and learning questions in their disciplines when in fact what departments and sometimes chairs and deans think they are there for is to provide service support to do the multiple tasks that need to get done in a department and that because they have this education label attached to them they seem to be the likely candidates and that was a pretty strong theme even though we kind of moved away from it. We kept sort of slipping back to that.

DR. SMITH: And the public face.

DR. FERRINI-MUNDY: Right, the state has an issue with certifying computer science teachers and so who is the person who needs to be called on to do that, and it is Mark apparently. So, just check with him and the notion that the research can be basic research and that research in these fields is not always about finding an immediate solution to some particular problem of teaching and helping people to understand but that is what some folks in this discipline want to work on and they aren't necessarily about finding the best instructional approach to a particular topic at a particular time but instead need to look at the basic underlying issues and so what I was hearing was almost a mismatch between what the departmental goals for these positions sometimes are and what people seem to really want to do with their professional work. It strikes me as kind of an interesting problem.

Another interesting observation about interdisciplinarity(?) within this area of STEM education research and the sense that sometimes the algorithms that are called for in funding in RFPs and so forth that say that we need one cognitive scientist and one engineer and one mathematician to be the team to do this kind of work that those don't always fit well with the kinds of mixed expertise apparent within individuals in this STEM education research field or a person might have substantial background in physics and in cognitive science and so it might not be so necessary to have a physicist and a cognitive scientist together and that I thought was kind of interesting and finally a very strong emphasis on the importance of mentoring on providing opportunities for students to get to professional meetings and to become part of the community. That came through again not surprisingly based on this morning's discussion about working in a community someplace.

So, I am not sure that we stuck exactly with the questions but a very interesting discussion and I tried to push a little bit about what about the connections to K-12; you know, that is an area where there is a lot of money, more money probably than there is for work at the postsecondary level where there is just a huge operation naturally that one might connect to. Is it strategically wise for some of the folks interested in STEM education research at the undergraduate level to be more connected to that and nobody seemed to be really excited about that. You know what I mean. It happens and it is part of the expectations. Certainly the connections to teacher education are difficult to avoid but it didn't sound like for the folks in our group at least that that was a main strand particularly in science education focus at least at this stage in the development of these disciplines. Does the 104 group have more to add?

DR. TANNER: Good job.

DR. FERRINI-MUNDY: Thank you. That was not what I was looking for but thank you.

DR. HILTON: Do you want to take some questions or do you want to go right to our next breakout group?

DR. ZOLLMAN; Let us take some questions now or general comments.

DR. WITTMANN: I find it fascinating that the next task is for us to talk about institutional arrangements and yet it seems that in our first breakout we spent a lot of time talking about institutional arrangements already and I am wondering if we should spend the next one actually talking about motivations. Actually Joan pointed out that there were motivational issues talked about in our group.

DR. LIBARKIN; I actually thought in coming to this workshop one of the outcomes for me that was very interesting was the concept that there might be some talking points and things that those departments that might want to create such positions could consider in creating those positions and I don't know that we have discussed what is it really that I find to be positives in my position or I find to be negatives in my position and for everyone here as well maybe that is something in the administrative conversation that we can maybe wind our way that way. I, personally, would be interested in that.

DR. HEPPERT: Just a real quick reaction to something that I walked in on at the very end of and so I probably will be saying something a little out of court. Myles, I very strongly agree with the idea that when we have learned lessons if we don't apply them to what we do in the next step then we are doing something fundamentally wrong and as a consequence it is really important for us to move to more quantitative measures, to better measures of the efficacy of everything that we do.

I agree with you completely there but I really have a visceral reaction when I hear the words and I hear the words more often than you probably think, when are we going to be able to declare victory and go home with educational issues; didn't we address these educational issues 10 years ago with a program or 20 years ago with a program? I once heard Dean say that he solved the same educational problems three times in his career and it probably a fourth time since we had that conversation.

The issue is nobody would ask when is physics going to be done or when is chemistry going to be stand still or when is society going to be perfect? When are we going to stop inventing new technologies that we might have to apply to education? Education being a human endeavor if anything is in many ways as we all know more complicated than some of these other issues that you can control and understand precisely. So, I think we need to in a very tactful way send that message to our colleagues.

DR. BOYLAN: I agree and I was making that an extreme statement just to illustrate the point.

DR. HEPPERT; But actually you do hear you know like a billion dollars has been spent. Are we done?

Actually those kinds of statements do come out of some of our colleagues occasionally.

DR. SIMMONS: But actually as a physicist I would remind you that at the end of the 19th century people thought physics was over and then quantum mechanics was discovered, and we started again.

DR. HIXSON: The problem we have at the NSF is hearing oh my goodness, pick your level but for us it is undergraduate education in science is awful. Well, that is the part that you would like to declare victory over. You are flagrantly awful.

(Laughter.)

DR. HIXSON: And if there has been a billion and still every proposal starts with who are we, then you are really bad. It is just attitude and semantics you know. There is this paragraph to put in there but that is a problem and it is beginning to come back and hit the field.

DR. HEPPERT: Haven't any improvements been made?

DR. HIXSON: Yes.

DR. PIMMEL: And how are you building on what you have already done, and I think that is the key. In physics, in any science you say, "Okay, we already know this. Now, we have go to know this," and we couldn't even ask that question before we did this. This same thing is true in education but the point has to be made and I don't think we get what Susan says. Like nobody ever did anything or I am going to change it like nobody else has; so, we have got to get away from that. Part of it is ego, but it is a self-defeating kind of approach.

DR. ZOLLMAN; So, Terry, are you saying that folks in disciplinary education research need to be sure that they are pointing out in their proposals and other places what they are building on?

DR.PIMMEL: Right, at least it got us this far. Now, we have these tools and we have this new challenge because the science has changed. The students have changed. The student population has changed. We now have better tools to solve the problems like somebody said, "Well, I already gave you a billion dollars. Why should I give you that $100 million dollars? You didn't solve the problem. You are just posing it."

DR. MC BRIDE: I think it needs to be clear that there is a body of knowledge on which any disciplinary education research can build and must build and on the one hand we don't want to reinvent it again at great cost. On the other hand we don't want to go off without, as you would not do in any other field go off without building on anything that has gone before and I think that is very important for the credibility of disciplinary-based educational research that it build on a body of knowledge, that it contribute to a body of knowledge and that there be some general agreement that we do know some things.

DR. PIMMEL: Even if it is reinvestigating what we have already investigated. We do that all the time in the sciences because you now have a better tool to measure with or the situation has changed.

DR. TANNER: I want to back up what Terry said about you do that in the sciences all the time and so there is this great new conclusion that people have that if there are no new neurons in the adult brain. Even people who work in biology walk around saying that. Well, there has been data in the literature for a long time suggesting that is not true and now some folks who have a little bit higher status than the original paper authors have brought that back to bear in the last 5 years and that is now a huge question in neurobiology and it is good that that has been revisited and I think that one of the things that we as discipline-based educators can do especially those who have been trained in the sciences is to try to bring this idea to the sciences, that education things don't get solved and that reinventing the wheel many times is about continuing to learn and that it is challenged more so because the molecule that I study in my lab is the same as the one in Tokyo but the students that I study in my classroom are different than the ones in the classroom next door. So, it is actually even more challenging and we see more revisiting in the field of education but somehow that has not been acknowledged by scientists and I think that it gets put off and claimed that oh those people in education don't know what the hell they are doing and really it is a very, very complex problem.

DR. RASMUSSEN: I agree with you but I also want to point out that, and I don't mean any disrespect to NSF but there is also a large value placed on creativity and newness in NSF and there is not necessarily a sense that replication studies are good. We want to fund it. We want to see the replication studies that I see from NSF. Maybe you have been reading the wrong RFPs but I agree with you 100 percent. There is a push for all of us to be innovative and creative and come up with new ideas and new models and new frameworks. This is a tension that we have to live with.

DR. BERGIN: I do think within that tension replicability which occurs in a new context, it is about advancing what we know and so replicability just for implementation's sake, NSF is never going to have that kind of money and that is not the role of NSF but replicability that advances, this worked here, and this is what we found out but when we do a year with this set of kids or this set of students or this is how we learn, absolutely and so we look upon that as pre-activity or information because it is taking it to a new level.

DR. GOULD: I was interested in the comment that you didn't get any resonance out of K-12 because I am thinking about this whole continuum, lifelong learning and waves of adults coming back for education, and we have now got the women in higher education but now we are going to get the minority groups in, too. So, there is a continuum of learners there and it seems if the STEM disciplinary people don't try to exploit the fact that that is the new playing field they are losing out a lot of opportunities.

DR. FERRINI-MUNDY: I would agree and I would also come back to this question about the accumulation of knowledge and advancing what we know and so forth. There is a base of research about K-12 STEM learning that gets attended to in very different levels of depth across research at the undergraduate level. I mean there are parts of the math ed world who do undergraduate research who absolutely don't cite the folks who are doing K-12 theoretical work or conceptual work and there are others who do and so I think there is an intellectual and substantive piece as well as a practical consideration around the K-12.

DR. ZOLLMAN; Yes, I think it is important to realize you know when folks are talking about building on a foundation we can't build on just the foundation within educational research because our foundations go much broader than that.

DR. GUZDIAL: In our discussion when we came up with the K-12 Maura made a good point about this is really about development and so for example those of us who are doing work in computing education are really interested in what is going on at even the pre-school conceptual level. How do students' understanding of process and time and agency influence their ability to learn computing later on? But there are just not that many of us doing computing education research and if we are all flat out working at the undergraduate level that is a whole new body of concepts and a whole new body of literature and new methods that have to develop to go further back.

So, I think it is a progression of the field to be able to explore even the K-12 area.

DR. LIBARKIN: That is where the field needs to talk to each other because of the things you just mentioned. It is a completely different question. How do I know how to get at your work, right? How do you know how to get at my work?

DR. ZOLLMAN; I think we need to move on.

AGENDA ITEM: Session 4: Break-Out Discussions of Institutional Arrangements

(Thereupon, at 2:20 p.m., a recess was taken for breakout groups until 3:15 p.m., the same day.)