DR. FERRINI-MUNDY: I think we are well into the content of Session 6, but yes, Session 6 is officially titled Next Steps for Educational Research Positions in STEM Disciplines and if you will notice on Page 5 of the agenda there are three questions that are there to help us with discussion where we see the first is how can or should the success of this movement be judged. We need to think about I think a question like that because it points us towards what sort of monitoring documentation, self reflection is necessary to really know whether, first of all whether this really is a movement. That is an interesting use of language, you know, is there a coherence to this that would enable us to call it a movement and secondly is it having impact and what would that mean and what would count and secondly what research is needed to understand the scope, institutional arrangement and future prospects for educational researchers in STEM departments and it is interesting to me also because it suggests the need to collaborate with other kinds of experts if what we really want to do is watch this because this would require I think research knowledge about organizational change and higher education change and you know that would create another line of interactions that might be fruitful and then finally the question about schools of education and disciplinary departments and what kinds of intersections would be important for supporting rigorous discipline-based educational research and researchers, so interesting questions.

We have talked about some aspects of some of this. We, also, see that Scott and Bob's names are here as participants in this session and what we have decided is that they have each thought about these questions and they have been observing this meeting and have some points that they would like to make I think about these but they are going to give relatively brief comments mostly meant to kind of spark our discussion not that this seems to be an unsparked group.

For Bob and Scott the challenge to you is to see if whatever you can say can sort of provoke reaction particularly around these three questions and next steps and I think that maybe a ground rule that I would propose for the last bit of conversation is that as others speak after Bob and Scott let us try to put things in the form of recommendation not necessarily formal recommendations but ideas about next steps for both the NSF and NRC.

So, Scott and Bob, do you have an order in which you would like to do this?

DR. ASHMANN: Yes, the curse of the last name beginning with an A. When I got these questions and thinking about these I guess I would like to focus more on the second one than the other two.

My thoughts were along the lines of the findings from the study that I presented earlier today were a part of a larger research project that dealt with leadership in math and science education and slide No. 3 that I referred to in my presentation dealt with the three large questions of this leadership study; how did currently recognized leaders become leaders; how do current doctoral programs serve as preparation for leaders are supply of and demand for doctoral graduates matched and it struck me that a parallel study could be done with this group of STEM researchers. How have people who have been in education research and STEM disciplines for a long time remained in that field; what obstacles have they found; what are things that have spurred on their careers along with the people who are just entering this career path and we have good candidates right here at the table of people who could be interviewed for early researchers in STEM disciplines and then looking at the doctoral programs that are producing these individuals as well and what are the opportunities for teaching or becoming involved with K-12 issues or undergraduate education issues? What opportunities are there for these individuals to learn the tricks of the trade so to speak if there are any of those tricks and then also looking at the supply and demand issues as well and specifically for education research positions in STEM disciplines and I think that this idea in some ways was supported after Chris asked his question by Dean and Joan in saying, "Let us study ourselves."

Maybe that is the thing that we need to do next is learn some things from what we have been doing thus far and see if that points us in a critical direction for the future.

DR. GLASGOW; And just to continue with that, that is basically the study that I shared part of the things I studied. That study was really about trying to see who are we as a field of mathematics education by looking at the product of doctorates, looking at people and specifically what they do.

So, you know trying to answer the first question, how can we judge the success of this movement I think first of all we have to decide if it is a movement and I think this comes down to it is similar to what my study was about which is you are going to have to start identifying people. It is based around people and that may seem like a simple task but it won't be because my eyes have been opened to a lot of different things.

You all are a different group than I have ever presented this to before and you have asked me questions I have never thought about.

There are all kinds of people who are doing content specific research. Some of those people are in my field, mathematics education with a big M or with a big E or with a big R.

I mean they are different people and my study was very focused on a certain set of people. I think you could expand that. I mean I am interested in going back and looking at dissertation abstracts and finding people now who were not PhDs in mathematics education; they are mathematicians who are doing research in education and seeing what they do.

So, I think you know the first step would be to identify people and that will take some collaboration to do that and then secondly letting those people show you what they are doing instead of trying to define what this is, see what they are already doing and looking at the set body of research that they have and then trying to decide well, is there a consensus here; is there a body of mathematics education or science education research that is out there that is being done and then going from there and that is where you can start to tell if there is a success.

I think the success will be more in that line in terms of a body of research, a foundation of understanding that is being built amongst a lot of different people in a lot of different ways and then also when you identify those people, look at their body of research, you can also start to follow up and find out okay what are the barriers; what are the things, obstacles you face, sort of what we have done here but opening that up and doing a bigger study of something like that, of what are the obstacles; what are the things that need to be changed for you to be able to do more in terms of your kind of research?

DR. FERRINI-MUNDY: I think now we are open for comments, questions and remember to think in terms of suggestions really for next steps. I saw Elizabeth and Kimberly.

DR. SIMMONS: So, in terms of trying to identify who these people are presumably you don't think that you could just sit and read all of the thesis abstracts. Would you have to contact departments and ask them to at least as a first step to identify what they think their science and math education --

DR. GLASGOW; That is basically what I did with mathematics education. I mean I did read abstracts. I read a lot but I eventually sent things to departments and said, "Tell me who this person is and where they are at." Just finding them was one thing, you know, just finding where they are at now.

DR. SIMMONS: I mean even to find them initially would you ask departments within the last 10 years who are the people you have graduated who did this kind of --

DR. GLASGOW: You could do that. I actually started that way. What I found is that we don't remember people as well as we, I mean we don't keep track of our people. We think we do but we don't always.

So, I gave them a list to begin with of all of the possible, you know, all the people who did dissertations related to mathematics in the last so many years.

DR. SIMMONS: Related to math or related to math ed?

DR. GLASGOW; Related to math ed, yes, related to math ed.

DR. SIMMONS: How did you generate that original list?

DR. GLASGOW: Doing a dissertation abstract search for dissertations related to mathematics education.

DR. FERRINI-MUNDY: That is problematic, right, because some of the folks that we might want to look at perhaps did a dissertation in mathematics or in a science area and later on their postdoc began to shift. So, it is more complicated.

Kimberly an then Chris?

DR. TANNER: There is a book that was published by an Australian science education researcher whose name I am blanking on and at the level called defining an identity is the first half and I forget the second half.

It is something like building the profession of science education research and that approach was an interview study of 70 leaders in science education and I forget how they were sampled and those 70 folks were asked to describe what they thought were their two most influential publications in science education research and why, and they were asked to identify two researchers that had most influenced them and their papers and why and I came across this book after an international science education conference in India about a year ago.

Somebody got up and waved it and I have found it really very enlightening in terms of accessing literature and so I just want to kind of highlight that because to a certain extent what Scott was proposing in terms of a parallel leadership study has already been done by folks who may or may or may not be identified as a discipline based science education researcher.

That said when you read that book it is clear that a lot of those people who are science education researchers in colleges of ed identify as scientists and so in the spirit of building on things there is a pretty recent publication that starts to address a very related question with maybe a slightly different sample of people.

DR.SMITH: What was the last part of the comment, that defining identity doesn't come up on Google?

DR. TANNER: Yes and this is why my memory is failing me. It is a beautiful little purple book by Peter somebody defining an identity.

STAFF: I think I found it. The Evolution of Science Education as a Field of Research: Defining an Identity?

DR. TANNER; Thank you, a really fascinating book and the bibliography is good for us.

DR. FERRINI-MUNDY: Let us just give Dorothy a round of applause.

(Applause.)

DR. RASMUSSEN: I was sitting here listening to everybody and very intrigued by the idea of having a series of conferences that we would invite a number of the different STEM education disciplines to focus on particular questions, say, for instance, well, I might as well back up and say that when I come to these conferences one idea would be to identify the various centers for research in math and science education across the country at different universities and my cohorts from those universities.

So, you get a sort of focus, target people together and I was just thinking of some different questions. One would be within your area if it is biology education or chemistry area what are the central problems that your groups are trying to address nationally.

The second question might be what does it mean in your discipline to understand an idea, sort of what are the cognitive underpinnings and then what are the sort of emerging methodologies that your discipline is identifying to document the evolution of those understandings and it would be fascinating to me to see where people stand across the different areas and I know NSF has a series where they fund conferences and one might sort of start there. It is going to be small as a pilot where you investigate that kind of conference setting on a small scale and then see where it takes you and just in relation to your comment about the evolution of science education there is also a very recent book in math education called, I think it is called the classics in math education.

They asked a similar kind of thing. What would you identify as the five most important method published research in the last 20 years and a number of people have identified those.

So, I mean that could even be shared readings where each of the different groups brings some collection of those readings to inform the others about the ideas.

DR. LIBARKIN: This I guess builds on what the last three people have said. First I will just give a plug for Michael since he is not here and at Maine they have a science education group and I want there a few years ago. I don't know what it is called anymore. They have changed the name but they are actually having an interdisciplinary science and math and engineering education conference and I think he is going to e-mail everybody and tell everybody about it because he wants to bring people out to have these kinds of conversations.

I think sort of the larger question that feeds into everything everyone has said is how can we build a community in STEM education research because that community for those people who work within disciplines doesn't necessarily exist and then sort of to be even more amorphous about it how do we build meaning; what is STEM education research; what is that?

I know in geoscience education at least it really has disparate meanings depending upon whom you talk to and I think that would be a fascinating research project to understand what we mean and it builds on what Robert and Kimberly and Chris have said to build meaning to community.

DR. FERRINI-MUNDY: To develop sort of a shared understanding of even what we are talking about.

DR. LIBARKIN: A shared understanding so that when we do come together and talk about STEM ed research we know what we mean.

DR. HEPPERT: I really do think that this is an incredible opportunity and a primary focus, this idea of building connections across educational research in the different disciplines but I think as we look to the future and if we really want to prepare for an explosion of activity in the future then there are some other kinds of connections that we need to think about making. An important one is between the educational communities and these disciplines and the other researchers within our own disciplines. This has been an issue that has been a constant concern and I think it has been a sort of undercurrent of discussion here is educational research really valued within the disciplines; is it really addressing issues that are critically important to the other researchers in our disciplines. So, I don't think we can, in the light of excitement about building these kinds of bridges I don't think we can afford to lose the idea. We have to build bridges to the other researchers within the disciplines.

Two other areas we have to build bridges and again I am not sure that these are necessarily as high priority as what we have already talked about but definitely building bridges between educational researchers in the disciplines and the education of researchers in the schools of education if you want to think about it that way, the capital "E" folks is a critically important issue, and I think that finally building bridges between educational researchers in the STEM disciplines and policy makers who are going to set priority and set value on what ought to be funded, what ought to be fostered in the educational disciplines in the future. Those are critical things and we could do a wonderful job in one and completely be stymied by not working on the others.

DR. LEWIS: Two things, one of them was as you said that I think that having the energy to build those connections with my chemistry research colleagues comes out of the connections I have with other educational researchers. So, I have to work on both fronts simultaneously or I won't be able to build those bridges with my colleagues.

What I was thinking about when you mentioned earlier in response to what Julie was saying about defining ourselves, chemical education research actually has several position papers, some really nice definitions out there and they are useful and I think they are useful in that they don't tell people that there is any one definition that is correct. There are several different ways to be a chemical educator. Those are all valid ways. Those tensions are always played out in the field. I actually think that that is a sign of health. I think the discipline of chemistry is healthy because chemists have declared it is whatever chemists say it is and whatever funding we need to get it is chemistry.

So, other than having a group of people articulate a few possibilities going too much further than that I think would stifle the field rather than have it be vibrant.

DR. LIBARKIN: To me part of building isn't to stifle at all but it is to bring acceptance from the larger community. I mean that is a piece of it. So, I know that some people have had that conversation that the larger community looks at the STEM ed research and says, "I can do that, too," because what they see is pedagogical publications which is in my mind viable STEM at work but how do we get all of the pieces that make up STEM ed research, how do we sort of define those pieces?

DR. FERRINI-MUNDY: I think something that we have not talked a lot about today but that maybe needs to go on kind of alongside the sorts of efforts that people are describing now is a much deeper look at the graduate preparation of disciplinary researchers. So many of the stories that we are hearing are from people whom we know of who have come into the field late after they have done their disciplinary preparation and who come in through postdocs or through kind of getting involved in projects and trying to assemble their knowledge in what might be an ad hoc way. It may be that a next generation could draw much more heavily on more formal graduate preparation and we have wonderful examples particularly in physics of how that has worked in major research departments but another kind of conference and there has been some of this work in math but not quite with the same focus. Another kind of conference might be on sort of what kind of preparation would be suitable for the next generation of STEM ed researchers who would be prepared to work in disciplinary departments and how could more of that be available across the country because right now I think it is still pretty limited. So, we could add that to the list.

DR. SMITH: Yes, maybe some nice ties to the, there are several studies of the doctorate right now. There is the Carnegie study of the doctorate that George Walker is looking at. There is the re-envisioning of the PhD. There is Woodrow Wilson study that is called The Responsive PhD. I don't know how much they have looked at STEM PhDs but there is a lot of activity in that area.

Myles, you have looked at PFF programs pretty carefully.

DR. BOYLAN: That is pretty independent of the single discipline but there is a national movement that would certainly be in tune with what we are talking about here. It would embrace the value of developing PhDs who specialize in education research on issues in the discipline and also good teaching. I am sorry but I can't avoid making that connection in my own mind. That is a major incentive for pursuing some of these issues.

DR. TANNER: I agree. I think that this issue of graduate preparation is really important and of all the things that are listed and I don't know them in detail but just recently looking at PFF I mean I feel like teaching has become more a central issue in those programs and even scholarship of teaching like Julie was sort of referring to but I don't feel like I have seen an intersection between PFF and re-envisioning PhD or any of those programs in discipline-based education research. I just haven't seen it and maybe I have missed it but I don't think that it is there and I think this issue of training I will just out myself, I feel like I need a boat load more training than I have and it is a limitation and I am trying to work with different people to really ramp up my methodological skills and what I have learned is that I have an enormous amount to learn and I don't think that my colleagues in the sciences necessarily recognize a lot of the work that has gone on before in social science and so I feel like today I hear these words like rigorous engineering education and the word "rigorous" is on the agenda several times and I am sitting there thinking all right, what does that word mean and in some conversations there are issues about quantitative versus qualitative and so I always translate it that you need to match your methodology to your research question but I know that that is a major elephant in the room that we haven't addressed of what is rigorous methodology; what are the methodologies that people are using around the table and what kind of information do you get from different methodological approaches be it think aloud, versus interviews versus surveys and if I were going to identify like a single barrier to my colleagues being able to recognize the value of education research it is methodological assumptions and so that training piece is exactly where those of us who are doing it could be better prepared to address it as well as sort of integrating it more broadly into the scientific discipline.

DR. LITYNSKI: I think some of the issues that have come up have kind of crossed between a science discipline and an engineering profession or a profession in general. Let us not say engineering. So many demands are made on people's time to help others is really looking at someone to be a professional in that sense of the system, someone in practice and I was jotting down a few things here and I spent time in both physics departments and engineering departments of various kinds and some of the preparation and kind of the thought that people prepare themselves I guess in going for each and some of the thoughts that came to mind and typically whether this is a movement or a subdiscipline or a profession there might be different expectations and ways in which you frame that.

For example, what is the mission slash purpose slash goal slash whatever of this movement slash subdiscipline slash profession? If you are a professional then there is a professional code of ethics and there is a professional society and all these things that go with it. Then there is a body that has put a lot in. Then you have a body of knowledge typically that goes along with whatever it is, whether it is a science or engineering or whatever, or a profession and then what is the practice of it. Now, clearly the professional is looking at the practice of it more so than for a scientist who is looking at the practice of what they are doing in that and not necessarily the practice of profession because in the profession you will be servicing customers typically whether it is medical, whether it is engineering or whatever and therefore if you look at what is it you are doing and how do you service your customers and what are their expectations and that is some of the tension I have been hearing I think around the table because as some of us go into various departments there are expectations that you are going to be servicing a group who are the rest of the discipline and they look at us as practitioners who are able to do that rather than as scientists who are doing research and so how do we define ourselves in that? Do we define ourselves as doing both or one or the other? And then if you are in practice what are the results of that practice as a part of preparing yourself and then I think the comment that came up, how do we purport ourselves for this: do we purport ourselves to be scientists? Do we purport ourselves to be professionals who are practicing or both? What are the measures of success in that? And they would be different depending on what you see for yourself as goal or a mission or whatever it might be. Those are just some thoughts I had as I listened to this conversation.

DR. COOPER: I was going to respond to the comment down here about the research methodology but I don't think they have to understand. I think what they have to understand is the results of our work and I think we have done a pretty poor job of publicizing the results of our work, and I don't think we do a very good job of telling our members of our department the results of our research that could have an effect on them because it should have an effect on them I think. Otherwise why are we doing it. If they are not going to change and we sit in our corner and do our research and collect our accolades or whatever then I think we have failed actually and I think that we don't do a very good job pushing it out in the outstream, that there are things that you can do to change the education of your students, your undergraduates on whatever it is that you are doing in your research and I know there are lots of people around this table who have done a good job of that but in general we are not very good at that and we are not very good at singing our own praises if you like because there are things that we know now that most people in our departments and my department as well do not know. They do not know or they do not believe it. So, we are not putting out evidence that they can understand and I think we have to do that, otherwise we are not going to be ever taken seriously and not that we are not taking seriously folks but I feel strongly about this.

DR. HILTON: I just had a question for Kimberly. When you were saying it was such a big barrier, your colleagues' assumptions about methods is it that your colleagues assume that your methods aren't very rigorous and are weak or something because they are not members of biology or what exactly are you saying when you say that your colleagues' assumptions about methods?

DR. TANNER: I gave an example earlier which was a recent colleague who is very well intentioned. He has taken a reform-based approach to teaching and said, "Let us put together a little survey and we will collect the data and we will publish an education research paper." I think there is an underestimate of how much work goes into doing an education research project just like any other kind of research project. There is an underestimate because the words are familiar because anybody can type a survey in Word. You don't have to have like a NanoDrop spectrophotometer.

(Laughter.)

DR. TANNER: Because the tools are familiar and the words are familiar there is an underestimate of what is involved in putting together a very nice assessment or interview question.

DR. LIBARKIN: That even goes into those people who are STEM educators. There are some STEM ed folk who don't understand the work that might go into it because we are not well trained and I will say I was guilty of that several years ago, not anymore, but several years ago.

DR. FERRINI-MUNDY: I was thinking I would like to turn the direction of the conversation a little bit. We have got 10 minutes left and I would like to invite our NSF observers an the NRC staff who are here if you have issues on your minds that haven't been thoroughly discussed enough today that you would like to hear just a little more about now might be a good time to let us know if there is anything that we haven't gotten to today that you would at least like to get out into the mix of the conversation and if not I see lots of other people ready to speak.

DR. PIMMEL: It appears that in science and math that the STEM educators are out within the department and in engineering at least the two models we have here they pulled them out of the departments and set them apart. I wonder if there is a reaction to those two models, the advantages of one over the other?

DR. FERRINI-MUNDY: I would just chime in on that. I think that in math and science education it is actually quite mixed. Our focus here has been on STEM educators who are based in the disciplinary departments but there are lots of schools of education that have departments where there appear to --

DR. PIMMEL: But within the college of engineering is there any example of a STEM education, math-science education department within the natural sciences?

DR. FERRINI-MUNDY: Okay, I see the question. That is what we are trying to form at MSU. So, we don't know how it will work but we have been watching these engineering --

DR. SMITH: We are talking about it at Minnesota.

DR. FERRINI-MUNDY: Are you?

DR. SMITH: STEM education research.

DR. FERRINI-MUNDY: In a college of science?

DR. SMITH: In the Institute of Technology which is math, physical science and engineering.

DR. FERRINI-MUNDY: That is a good question. Do others want to chime in on that?

DR. SMITH; There is a big difference between engineering and physics, chemistry, mathematics because we don't have much of that in the engineering ed school.

DR. ZOLLMAN; That is because ed schools tend to focus on K-12 where the disciplines can talk to the K-12 folks because there is kind of a continuum there where there isn't that with engineering.

DR.PIMMEL: It is changing a little bit in some states, for example, Massachusetts.

DR. FERRINI-MUNDY: Particularly as teacher education issues come into play, you know if there are spots in the K-12 world for engineering curriculum to make its way in, you get this phenomenon of needing teachers and then you get a crossover.

Myles had his hand up. Yes, Myles?

DR. BOYLAN; I just have several observations and comments and going back to the work that you two guys presented to us I get the very distinct impression from reading what was in our notebooks and listening that you would say that right now we are not oversupplying the country writ large with people with these skills because there seem to be ample employment opportunities for them. Is that a fair conclusion?

DR. ASHMANN: I think it is more so in mathematics education than science education. I think there is not an overabundance in either one. I have heard anecdotal stories of institutions that will post for a mathematician and receive 200 applicants and post for a math educator and receive five. So, I think that that disparity might not be as great in the sciences but it is still is not something where we have an overabundance of science educators,

DR. BOYLAN: The other comment was in the Division of Graduate Education in one of our major programs there is the so-called "graduate research fellowship program," and these are supposed to be nationally prestigious scholarships, $30,000 stipends for 3 years, portable to any institution and a fair number of them are given in math and also a number in fields of science and I have had the privilege of reading some of the proposals that the students wrote and the impression I walked away with is these are very focused on making a real difference in the world not in being the best researcher just because that is the macho thing to do.

Many of them are focused on making the world a better place and I was wondering if you ever thought about taking some sort of winners in doing math and polling the award abstracts of those people to see how many of them seem oriented in that direction, maybe from 10 years and then seeing how many are actually in that area. It might be an interesting thing to do.

DR. ORLAND: Since I was not here for three-quarters of the day it is hard for me to say anything. So, this can be cut very short by saying, "We discussed that and dealt with it." I guess I had two thoughts about what may or may not have come up as potential barriers to the challenge of having STEM researchers in disciplinary departments more accepted and integrated and proceeding in a smooth glide path within their disciplines which I think we all agree is something that we are aiming toward and would have great value.

One which I guess I mentioned in a brief side conversation with Eugene is the extent to which the, what seeps out in terms of the perception of others in the disciplinary departments is the reputation of the schools of education within that same institution such that if someone is seen as doing education research within the discipline because it is education research and because I have some familiarity with what schools of education do and they are not terribly impressed with the quality of that, that then becomes part of their mind set regarding what goes on in what you are doing and let us face it we have serious quality issues in research in schools of education.

As a matter of fact what I am hoping is that if this movement really does congeal and become something it can be the leading edge to enhance quality throughout educational research. That is my dream down the line. So, that is one potential barrier.

The other one is and again I am just laying these out as hypotheses, but my own experience in higher education and my field is political science, by the way, is the extent to which there is a focus on the quality of teaching and teaching is seen as something that is subject to canons of scientific inquiry and there are better ways and worse ways to do this, etc., and that is the essence of what you are looking at then that can be perceived as personally threatening for the people who are doing their jobs teaching because they have never really looked introspectively in terms of the quality of what they do. They were not particularly hired with that in mind and that may lead to some realizations on their own part, well, maybe I am not particularly well trained if in fact there is an evidentiary base behind what is more effective or less effective with regard to my teaching practices. Again, I don't know the extent to which either of those may play a part in the difficult cultural chasm that we are talking about here but I did want to lay it out because those were some thoughts I had which may or may not have been discussed during the day.

DR. FERRINI-MUNDY: Reactions to Marty?

Yes, Joe?

DR. HEPPERT: That latter one comes back to I think Terry's, and she is not here anymore but Terry's very well-reasoned argument that you don't always want to start every educational proposal with how qualifying things are and how we haven't made any progress in the last 20 years. It tends to eventually reflect back on the field and you know while we may believe firmly that we know more about good teaching practice and know more about effective teaching methods than we did 20 or 30 or 40 years ago you never need to start a conversation with colleagues from the perspective that boy, are you a crappy teacher or as Melanie said, “Haven’t you heard anything over the last 15 years?" You don't need to start the conversations that way. You can start the conversations from the perspective that we can always do better. No matter how good we are doing, no matter how effective our educational programs have been over the last 40 years and it is awfully hard to argue on one level of measurement or maybe even many levels of measurement our science, math and engineering education routes haven't been effective over the last 40 years. We could always take the perspective that we can always do better. Can we reason together to figure out how that is? So, I think it is more a perspective of how you approach things culturally and it is very important for us to learn those diplomatic skills.

DR. FERRINI-MUNDY: We have about I think a minute left because people are worried about snow and cabs and so forth.

So, I guess at this stage I would see if Dean or Karl have any last words they would like to offer as part of the low-key steering committee here.

DR.SMITH: Just the idea that there is always room for improvement. I ponder that often. Why is that so difficult? Engineering is always changing the state of the art and it is just the way we do things but in engineering education it is not the norm and part of it may be the benefit/cost ratio. People say, "Oh, my gosh I could do things but at what cost. You know I would have to learn something new that is really hard."

DR. HEPPERT: Or there will be an opportunity I could take someplace else

DR. SMITH: Yes and then other thing that is really interesting that hasn't come into our conversation much at all today in the private sector companies are hiring folks to study how is work being done; what really is effective; how do we acknowledge workers; what are we going to do if this future group of people are going to retire which is an opportunity we faced, too, I think in the Academy. Look at the age distribution of engineering faculty for example. It is a time where we really need to figure out what expertise they have and how to capture it.

DR. ZOLLMAN; I guess just to follow up on that in terms of just interacting with colleagues Joe's and Karl's points were good ones that if we go in like a bull in a china shop telling them all what they are doing wrong we are not going to have much effect either in making changes or keeping our own jobs except for those who have been there too long to be fired and there are lots of changes that can be made and I think in our last session Julie said something about changes going slowly and of course everybody reminded her that after all she is in the geosciences.

(Laughter.)

DR. ZOLLMAN; So, it really does move at geological time scales and sometimes we just have to be satisfied but frustrated with that.

DR. FERRINI-MUNDY: And on that note then I guess I would say thank you to everyone. I think this has been a fascinating discussion and I am much more optimistic actually than I was coming in that there is a movement and it can grow and it can thrive but it will take continued strong conversations in the directions of the kind we have had today.

So, thanks, everybody. Safe travels and unless there are other announcements I guess we are adjourned.

(Thereupon, at 5 p.m., the meeting was adjourned.)