DR. WILLINSKY: The first thing I am going to do is stand up. It has been a long day. I don't know about the rest of you, but it has been a long day for those of us sitting in the back and if you notice my neck kind of bent like this, that is because one speaker was sitting on this side of the podium and one speaker was sitting on the other side.

So, I thought if I just stood up and faced you we could maybe get started again. I am here to talk about technology and there will be no PowerPoint slides, but we are going to do some live from the web, and we tested it out; so it is working, but without the PowerPoint you are going to have to listen carefully and for reinforcement you can watch my lips.

The questions about technology have come up repeatedly today and I want to turn them on on the question of quality and the question of access but I want to address at least at the beginning some of these particular issues people have about the machinery.

In fact, it even talks in our little blurb about hardware and software. There will be no references to hardware and some slight ones to software.

Gary and I are both in the field of developing journal technologies if you like or online journals, and in fact, Gary is going to do his part but this is more than an online journal, but we both have been involved in developing the online aspects of research, in fact access to research. We have both been involved in developing systems that will support that access to research. We have in a sense gotten involved in the software development field and it is a regrettable step but it is one that we have taken and it is one that speaks to almost, well, I would hope many of the issues including issues of mediation, including issues of indexing, including issues of access to this knowledge and so the theme for me after listening to this, I certainly had prepared some ideas on coming here and then I started to listen to the audience and realized that many of my concerns about public access needed to be framed in terms of quality and of access in a larger sense.

So, the question of access to research for me which hasn't come up in a pointed way particularly around access to technology and access to research in terms of open access or free-to-read journals and fee-based journals is a point of quality in terms of the circulation of knowledge.

We need to think for a moment about quality of knowledge as having as lot to do with its circulation and its ability to circulate for me is one of those determinants. The less it circulates or the restrictions of its circulation have the impact of reducing its claims to quality, I would argue and so just very briefly a little historical moment. In 1965, the first journal begins. It is a 16-page philosophical transaction, almost like a pamphlet and Newton, Isaac Newton gets wind of it at Cambridge and decides that maybe this is a way to go.

He waits a few years and he is reluctant to publish. Newton is the most secretive of scientists and in fact this was his one venture. He only has one published article and still did okay.

(Laughter.)

DR. WILLINSKY: But what is important about Newton is he realized after 3 years of making notes from the philosophical transactions that he needed to give it a try but as much as he abhorred any kind of public attention any kind of public engagement even with other scientists he realized that it was important that his ideas circulate and there may be some value in that.

So, he published, he allowed, excuse me, he allowed a letter of his on optics to be published and he allowed that letter to be published because of a very active editor, the great, great grandfather of all of the editors who have come here, Henry Oldenburg, and Oldenburg convinced him to publish it and he did and he said what followed was tedious because he had to respond to a large series of letters of concern that were addressed to him. Everything was in those letters in those days. The letter was the beginning form of the article as many of you know.

After 4 years he wrote to Henry Oldenburg and said, "Enough. I will no longer respond to any letters on optics." In fact, he waited. He published his book on optics only after one of the letter writers back then had died because he was so concerned.

The issue for me though was that Newton sworn to secrecy on many issues, decided that the access to this knowledge and that philosophical transactions was really a public document; the real business of the Royal Academy was held in chambers like this and the philosophical transactions was an offshoot. It was not its official business. It disavowed for hundreds of years any official connection to the philosophical transactions because the philosophical transactions was a public engagement and it risked the tediousness that Newton had to face by responding to letter after letter after letter of concern. He wasn't clear about his experiments. In fact, it set some of the standards, the very things we have talked about. It wasn't sample size. It was prism size that was critical in terms of setting up the experiments that Newton was doing with optics.

So, I want you to consider then that the issue here in terms of your advice, this CORE group's advice on quality of access, excuse me, quality of research is largely to do with this future, not with what we doing now, not with the qualities we are trying to maintain but the future and that future has everything to do with the circulation of knowledge because we are in a state of declining access in many institutions.

At the university of British Columbia we are 32nd this year in the ARL list of university libraries. At the University of British Columbia we have got thousands of journals. Now, we are beginning to build up again because of the bundling and the aggregating by Elsevier and the big corporate publishers, but we have cut journals. Has anyone else out there had that experience at their institute?

One, two. It is very good down here, three, four, okay. Yes, we cannot take for granted that the publishing that we are going to encourage and the quality and increase in publishing we are going to encourage is going to result in the circulation of knowledge that is critical to its claim to truth or claim to knowledge. We cannot take that part for granted. So, we need to be concerned about access.

Now, what has happened in publishing is we have as it were this insurgent movement, this bit of revolutionary radicalness raising its head called open access and what I think you need to consider in terms of the future of access to research is this quality of how open it is.

Now, there is open access completely free, Educational Researcher is a good example of that; the one journal that is online from AERA is free immediately and available around the world, but the rest of AERA journals have not yet made that decision or at least haven't made that step to become available online.

So, we need to think that the very move to going on line is not a question of preserving quality. The majority of journals, like about 80 percent of the journals now are online. APA has been a leader in this field of putting journals online and there hasn't been any question of quality, any question of loss, and in fact it sounds like the readership may be going up in terms of Gary's remarks earlier today.

So, there is a sense then that putting journals online is an aspect that is going to happen. It hasn't happened with books. It is happening with journals but the other aspect of that is who has access because of that and we are now part of a global economy around knowledge. We have universities distributed around the globe and those universities do not have equal access. They do now in medicine because of the medical program through the World Health Organization there is now access to thousands of medical journals available everywhere in the world because of a program in which the major publishers have donated if you like or provided free access because it really didn't cost them anything to register a few ISP addresses in order to allow universities in Kenya and South Africa, India to begin to use the journals and for free.

So, we are seeing then this aspect of openness and let me just give you just a quick primer on it in the time that I have remaining. The quick primer is the New England Journal of Medicine. Familiar? After 6 months it is open access, okay, a delayed open access. Their 25 million gross a year, I don't think has been affected by that 6-month delay. The access to knowledge that they are providing has been greatly effective

Other examples of journals, of course, the Educational Researcher, Teacher's College Record we will hear about in a moment and in fact I have been using some of Gary's figures in terms of the great increase in readership. The one study I want to leave you with though in terms of what is impact, now that we have done ISI, the ISI quality of impact factor and citations, the one study we have is from computer science and it is based on conference papers which we heard this morning were the highest quality in terms of rigor.

Steven Lawrence has done a study. He published it in Nature which is not an open access journal. It is a very expensive journal and he found that the factor was four to one, four times as many citations comparing print, conference papers and online free conference papers four to one. Now, I know the new standard here. You don't have to tell four to one on which side.

(Laughter.)

DR. WILLINSKY: That is the new standard. Just leave that hanging. It is in the article. Don't just read the abstract. It is four to one in favor of open access, of course. The readership is that much greater and Gene Glass who should be here in my place especially because of all the credit given to meta analysis in the previous presentation, but Gene decided not to come and created a spot for me. Gene Glass has worked with the Educational Policy Analysis Archives Journal, EPAA, has demonstrated not only that he is getting a much wider readership, 2500 a day people coming into his site but that he is getting 1 percent of those readers are journalists; 16 or 14 percent of those readers are teachers; somewhat less than that, maybe 8 to 10 percent are parents.

So, the aspect of public access is where I really want to land for a moment. This access of reaching the public, now, the mediation question, my position on it, we have all kinds of positions on it, and my position on mediation just to balance things a bit is we shouldn't do it, is that we have a place for it but we shouldn't do it because it is too expensive and because we don't know yet if we really need it. We shouldn't do mediation because we have seen a health revolution take place and that is the term of the Pugh Foundation, not my own. A health revolution has taken place because of access to research on the Internet, access to medical and health information, some of that research and doctors are beginning to change their practice because patients have access to information they didn't have before, most of it unmediated and that access is beginning to change not only the medical practice; we are called it shared decision making. I thought that was a very clever way of the doctors to pick up on that. Instead of the tyranny of expertise they have now moved to shared decision making but it has also changed other areas as well. I just want to pull a clipping, one of my favorite sources here in terms of analysis of events.

This is from last night's William Safire, the Age of Liberty from the New York Times in which he talks about a speech of Nixon's. Excuse me, Bush's great speech.

(Laughter.)

DR. WILLINSKY: I haven't been to the States for a while. Things change. Bush's great speech apparently was held last week. That is George W., and what Safire does is he begins to analyze it and then he says in the middle of the article no, you analyze it for yourself. Nowhere is it available except on the New York Times web site, and he read it over and over again and he claims or at one point he talks about reading summaries and excerpts and critiques lets editors and analysts do the thinking for you. Those snippets and applause lines won't help you grasp the import that you should have even if you want to disagree and he talks about it then and finally at the end he talks about the speech a bit and then he says finally, "But let me not join the summarizers. Invest half an hour in reading this moving exposition of the noble goal of American foreign policy." At any rate that is Safire's position, however it is but the difference for me is critical. The difference is you go check for yourself; you check the stories and don't rely on commentators and columnists and that culture has just begun. We have just begun to have that expectation.

So, how do we begin to address it in educational research? How do we begin to see it as a quality issue, that the wider the distribution and circulation not just in the United States but throughout the world will contribute as a check, as a critique, ass an aspect of participation on a much wider scale?

Now, in the time remaining to me -- how am I doing on time, Joe?

DR. TOBIN: Ten minutes, 30 seconds.

DR. WILLINSKY: I briefly want to show you the kind of work that I have been doing that really compliments the whole aspect of electronic publishing and Gary's as well in terms of what can we do.

The issue is for me I have discovered is don't do it yourself, that there is much expertise out there and that a Committee like this is in no position to begin to judge these technologies but it is in a position to help focus how these technologies address questions of quality.

So, what I have done through this public knowledge project that I am working at at UBC and very much what Gary has been doing with his web site is looking at ways of increasing public access, facilitating that but not by mediating, not by reworking it.

DR. SCHNEIDER: What do you mean by that?

DR. WILLINSKY: Oh, by mediating?

DR. SCHNEIDER: Yes.

DR. WILLINSKY; Oh, sorry. What I mean by mediating is the wonderful work that Judy's group is doing. The Cochrane and Campbell organizations are doing mediation and ARA now had some debate about whether it is points or bullets. Bullets was the first and they decided that wouldn't go well. So, they decided to go to points. So, that is what I mean by mediation and there is a place for it but Cochrane and Campbell will tell you, have told you that it is very labor intensive, but it is not the role of researchers to do.

So, what if we think about unmediated research and what we have developed. So, what we have got for the public knowledge project is a context. When we put research up on the web we have often thought about it as a paste and glue process. We will stick the research up on the web. If you look at Educational Researcher it is identical. It looks like it has been pasted on your web. In fact you can't read it on the screen because it is in two columns like it is in the magazine itself and you have to go down one and come back up the other. So, we need to look at the environment, and we have a lot of people who are beginning to do this and certainly in my field of literacy education we have a lot of lessons that we can bring from literacy education over to the posting and information context if you like or informational environment in which we post research. So to give you just a quick demonstration of that what I am showing you here is an research article, a real research article in a dummy journal by Theresa Rogers my colleague at UBC.

This is a journal that we have up on the web that is just a demonstration journal to show people the technology. The principal concept behind this technology is it can help you manage and publish journals and the reason that we built it, we used about 100,000 Canadian, 73,000 today, 65,000 two weeks ago in US dollars. We built it on the basis that we would help open access publishing. We would help journals publish in a way that would make more of their work open if we could reduce the cost and so we built and open source system that we distribute for free that helps to manage the review process, that helps authors upload their work including the supplementary files, their data sets, that enables reviewers to access them by comments all on an online basis, that allows editors to do a lot of their editing in airport kiosks and lounges because it is we access and it allows international teams of editors to begin to participate in journal publishing where we haven't been able to do that before because of the need for centralization.

Wherever there is a web browser, an editor can begin to do his work. Our system is as I said free to download. It installs on a server. It is based on the notion that the author or excuse me the editor and the author only know how to word process, how to fill in web forms and the rest of it is based on the system. What is the title of your journal? What is the title of your article, and I will show you some of the indexing aspects to it.

So, one feature we built into this is what we are calling the research support tool. For those of you sitting at the back it is just a blur of red. I realize that, but there is actually a text here.

What it does is it appears with every article published using our system, using the open journal system, we have an open conference system as well but this isn't an informercial(?). This is a demonstration of public access. What it does is it addresses the needs of the public for a context. The public has an interest in research despite some of the comments that have been made including the professional public, that is doctors and teachers have an interest in research when it is directly concerned with what they are doing, not every day, not every week but when there is a critical issue for them whether it is about their own health, whether it is about a new program they want to place in their school, whether it is a program they want to remove from their school, they have an interest in research and they want the whole story.

So, this context that is missing, they have the interest, the motivation as we say in my field. They have the motivation. They don't have the background, the prior knowledge, the context.

This device is intended to do that. I am presenting it to you only as an example, as an experimental example. We have been testing it with policy makers in Ottawa. That is still just an example, a prototype if you like of the possibilities and what I want you to consider then is the possibilities in terms of this question of quality of access to the public and quality of access for researchers.

So, what this device does and maybe at this point I will sit down, is it provides the reader with a series of pieces of information.

The first thing it does is it tells the reader that it is a peer-reviewed article. I don't know, I mean we are so ingrained with that we can smell a peer-reviewed article at 20 paces. The public doesn't know that and we insist that in every article it be named. It could be an invited article. It could be a non-refereed book review. It could be a conference paper, and we have an explanation behind that as you can see. There is a little clip of it.

Let me go to the meta data. First of all there are a number of devices like Capture the Site. This ties into Reference Manager and End Note for people who don't have time to even read the abstracts. They need to just plug that into their work. We are sensitive to that, but what we need to understand is that there is a new standard emerging for indexing and I want to draw to your attention Dublin core. You will hear that again. It is a standard for meta data. Instead of talking about indexing on the web they tend to talk about meta data. The Dublin core, Dublin, Ohio, kind of spoiled it for me but may be good for you, the Dublin core is a series of 15 indexing terms or meta data terms that run from the very predictable title all the way down to rights including a lot of, and I will come back to some of the other ones. What we have done with the Dublin core is we have translated that back into journal concerns and we provided examples from this very article that you are looking at. So, the public can very quickly see how the article has been indexed, who sponsored it? The American Tobacco Federation. Well, how has this been handled in terms of the sources and how has it been set up in terms of its publishing history including the data and other aspects like that, but then all of the information here is provided from two sources automatically, well, not automatically. I shouldn't say that, two sources at very little expense. One is the author. Now, the author is not the best indexer in the universe but the author is someone to index and the journal itself. The journal automatically generates information like the URL, the data, the fact that it is peer reviewed because it is that peer reviewed section. The article is indexed immediately as soon as it is published. We talked about lag time this morning. Think about the lag time of ISI, the web of Science. If it is 6 months for a journal, it is 6 months to a year for the web of Science and indexing is a critical issue that we repeatedly come up to in terms of access. The information that it provides includes the methodology. Judy was that, no, it was Hannah; we are indexing methodology. Sorry, didn't mean to disturb you.

(Laughter.)

DR. WILLINSKY: We are also talking about the type in terms of peer-reviewed literature. So, the idea that you can do a search on peer-reviewed literature, I don't know if you have worked with Google at all, that is sometimes an issue trying to find good literature.

All of these terms then can be indexed using what is called a harvester. It is meta data. I kind of like the terms. I mean the Dublin, Ohio kind of fits when you think about a harvester going along. What we end up doing is we harvest the meta data from all of the journals we are involved with and we in turn are harvested by University of Michigan's Oaister, O-a-i-s-t-e-r, because we are all falling under what is called the open archives initiative, OAI of meta data standards and this is all free. This is a free indexing system. It doesn't yet have a citation base to it but it will from a group called psych base in the UK and there tend to be some other aspects in terms of indexing.

Part of the context would have to be related to studies. One of the things that we want the public to understand is no study is an island unto itself. Each is a part of the main. Remember John Dunne's famous -- it is cited very often. So, what we do is we take two key terms from the author. What better way? A novice reader doesn't know about ERIC, doesn't know what key terms are but can very quickly go to ERIC and do a search and come up with related literature and so before the parent or the teacher takes the study to a school board meeting they decide they had better check to see if there are any other related studies or are they out on a limb on this. They could also though just as easily look at some of the instructional implications because this article is fairly theoretical, does talk about instruction. They could do a search on it. Do you know Marco Polo, the national organization of teaching resources? It has 66 resources on teaching about the Holocaust including some on Anne Frank which this article deals with.

We, also, have sites on media policy, on government policy, excuse me, that goes to first gov. Are you familiar with first gov? A wonderful resource that taps into not only the Federal Government but all of the states and does a search and again these terms can be changed if you find they are not doing an adequate job and there is another part of the article you are concerned about.

Similarly if there is a term that troubles you in terms of terminology you can double click on it and it comes up and you can do a search on it.

Now, let me just leave it at that in terms of resources. This is an experimental tool. The concern for us, let me just go over this one more time is that the circulation of knowledge is related to the quality issue. That circulation is being highly affected by online access. We are still in a state though, online access has not reversed the serials crisis. Do you know the serials crisis? Every morning you go down with your children to get, no, the serials crisis is the access to knowledge. the access to knowledge in terms of prices and in terms of the library associations still persists because the move to online publishing has resulted in a print plus situation. At UBC we are trying to get around that and many other universities by canceling our print subscriptions and we are saving all of 10 to 15 percent which is kind of humorous because it is much cheaper than that, the publishers, we are saving mainly on storage and processing by going to electronic.

So, we are moving to electronic and the issue that we need addressed by organizations like CORE and the National Academies and the National Research Council is the issue of how that translation or how that migration to online publishing can address questions of quality and those questions of quality for me hinge very much on increasing access that is both professional and oh, let us go three ways, that is scholarly and professional and public.

Thank you very much.