DR. KELLAM: It’s a great pleasure to be with you, the argument going on in education research about randomized field trials has been watched with a great deal of interest by people like us who get our funding from the National Institutes of Health and have for 35 or 40 years. And you might ask why would the National Institute on Drug Abuse fund this kind of study, augmented by the National Institute on Child Health and Development, that may give you a little bit of a clue, and I’ll try to explain that as I describe in my 20 minutes something about the scope of the work in three generations of preventive trials, randomized field trials, done in Baltimore City Public School system. And then Linda Chinnia, who’s our longstanding and wonderful colleague, will talk about what’s in it for the school district and why they let us randomly assign everything in sight over these years.

So my next slide shows what the cast of leadership is in the Baltimore City Public Schools. The reason I put that up there is not because we want to acknowledge their contributions because the partnership that we developed with them consists of us learning their vision of what exactly they want to have happen in the Baltimore City Public Schools over the next three to five years. This is the crowd of people who run and make decisions, develop the master plan and so forth, that we have to work to learn what they want to accomplish in the next three to five years so that we can integrate what we need in the way that helps them accomplish their mission. And I think that that’s been a theme now in the prior two presentations and certainly in Judy’s earlier talk this morning. The fundamental foundation of a randomized field trial is the partnership of the people who’s institutions you must work within, and when I was at the School of Hygiene and Public Health, it’s old name, at Johns Hopkins we used to teach that the first rule of public health is you shouldn’t get kicked out of the community.

The next slide shows you some of the cast of characters who did the work, Judy Peduska(?) is sitting here as our deputy director and actually the project director for the current generation of randomized field trials in Baltimore. But you’ll notice that there’s a bunch of characters on there who are in biostatistics and all kinds of other areas, there’s epidemiologists, Gene, myself and others, but there are biostatisticians, people who have done randomized field trials in other parts of the country who are also collaborators. The point of this slide is that in fact they all look to buy into this partnership concept but that the substantive issues in prevention science in education research, you would add, are as in need of methods research as they are substantive research. And as we’re doing work on the substantive issues of what works for which children under what circumstances the methods by which we do those studies are an equal arena for study. We’re talking about growth dependent modeling, we’re talking about implementation measurement, mediation, moderation, complex measurement and analytic issues. When you say it should be theory based it gets pretty complex when you try to draw the model and I’ll show you ours in a moment.

Well over the last four decades a lot has been learned about early risk factors, some of which you may or may not be aware of. If you’re in the public health field or the drug abuse or the HIV prevention fields of that sort you’ll know about these things, but these are some of the risk factors. Academic failure is in fact highly associated with almost all of them, if not all of them, so we have a mutual self interest as my mentor Sol Lalinsky(?) would put it, in joining forces. Concepts and methods from public health and prevention joining with concepts and methods from education research. Many of these risk factors have been subjected to randomized field trials and there have been a whole set of summaries of the power of the rigor, of the kinds of designs, some of which have been perpetrated on the public by people like me. So we know a fair amount about the risk factors and we know, or are beginning to know a fair amount about the kinds of designs. It’s not just one design, I think both earlier speakers this morning talked about randomized field trials is not one thing, it’s a set of things, set of design possibilities.

The one that I think is important in our work and may have importance to education research is what we’ve called the developmental epidemiologic trial. This is one of a set of strategies, you find early risk factors, like early deficits in reading, for example, and you attempt to intervene and you attempt to change that early deficit and shift the growth development, the trajectories over time to avoid an outcome like school failure. Well, the National Institute on Drug Abuse is interested because school failure in fact is a trajectorily highly parallel with drug abuse, and indeed with major issues of depression and antisocial personality disorder. That’s the reason for this congruence of interest among NIH’s and the Department of Ed, particularly IES. We share a great deal of the same developmental trajectories, and you can see what the issues are, you see if you can shift it, you follow the trajectories over time, growth modeling is an important byword, it’s not just a piece of jargon but really has to do with following kids over time and continuities and discontinuities.

I want to focus a little bit on the first generation of trials, which was in 1985, ’86, preceded Gene by two or three years, because I want to show you something about the long term trajectories we’re talking about. This was the design, there were five urban areas, three or four schools in each, they were matched. We did then random assignment of schools through one of the two interventions, one reading, the other classroom behavior, but we then also randomly assigned teachers in classrooms within schools, we had internal controls and external controls. Why’d we do all that? Because we didn’t know how to do it right. In the mid-‘80’s we didn’t realize how much contagion you’d get with in-school design, and so we did match external control schools. It turned out that the internal designs were far more efficient power wise, current design for example in the third generation we’re using 12 elementary schools for the randomization, it would have taken us 36 schools if we had the same design with a school based design. And indeed, David Myers went through the same considerations, where is he, in the Haan Study, which AIR does have some role in, at least in arguing with everybody, so the within schools versus cross schools is an important issue.

After random assignment in that first generation ten weeks later we looked at the mean aggression scores, one of the early predictors of drug abuse and highly associated with poor achievement, these are classroom means and you see the distribution of the aggression along the bottom, and the numbers of classrooms, the 40 classrooms involved up the side in terms of numbers of classrooms, and you notice this is very striking bimodal distribution. About half the classrooms lower then the mean in aggression and the lower half above 1.8 or so are in fact highly aggressive chaotic classrooms. I want to show you what happens if you’re in, next slide, if you’re in the top quarter of kids on aggressive disruptions in classrooms and you’re in a high aggressive classroom your chances of being highly severely aggressive and disruptive with all kinds of problems by middle school are 59 times the average child. If you’re in a low aggressive but you’re in the top quartile of aggression yourself, if you’re in a low aggressive classroom, 2.7 times the average child. The random design allowed us to make that inference as a classroom effect because we’re controlling, it’s a within school randomization, we’re controlling on families, communities, school building environments, so we can look inside the classroom and see what happens when you’ve distributed the first graders across the cohort.

Now this intervention we tested in the first generation, the Good Behavior Game was invented in University of Kansas about late 1960’s, it’s had a history of replication, not very rigorous but 13 or 14 studies at least. It’s geared to providing teachers methods of increasing socializing of children, improving and also to reduce the kids aggression. We did this intervention, I want to focus on the Good Behavior Game just for illustration, I’m not going to be focusing on the reading instruction arm of the original design. This is what it consisted of, it’s a group contingent classroom management technique, many of you may be familiar with, teams are rewarded for each kids behavior, if kids don’t behave the teams don’t get rewards.

Now we follow-up these kids nine time points through elementary school and then again at age 19 to 21. And what we find is a very marked interaction in the impact with the baseline risk factors, so kids who are highly aggressive in first grade were very vulnerable to chaotic classrooms and indeed the most likely to be benefited by an intervention in the first grade classroom. The reason I’m pointing it out are two things, one is these were quite shocking results for us. We were not expecting to find a long term impact from a first and second grade intervention particularly, although we hoped. Secondly, the magnitude of the impact was really quite striking. And third, the variation in impact we’ve learned to accept as a fact of the game. I think Judy maybe mentioned that, that in fact one gets into variation and impact, what works for which kids under which circumstances. We’re talking about multi level modeling in which the randomization is critical to being able to control a lot of extraneous stuff including in a within school design, controlling on school building and family differences and community differences. In any case these impacts were quite striking to us and there are others, more years of school attainment, actually Gene just did this analysis of the services, read services including special education but a handicap, which is a huge drain on school district budgets when you think about what the school district is getting out of all this.

So here’s some of the lessons. First grade, you all may have known this, we were shocked to find that, my mother was right, first grade class really are vital and indeed what happens to kids early on in socializing them is really vitally important to their careers. Then a large portion, epidemiologically, speaking about half of the teachers in first grade in Baltimore, judging by the data, needed better tools for classroom management, it’s not even required by NCAT last time anybody looked that we know of for accrediting schools of education to train teachers in classroom management.

So anyway prevention science in education research need to be integrated. There are concepts in public health and in public education that need to come together. We have a great deal to learn from each other and I mean that reciprocally, not one way. Most public health --

-- [End of tape.] --

-- Linda I’m sure will talk some about that.

Our goal here is to do three components to improve reading instruction, improve classroom behavior management, improve parent/classroom partnering. There’s a coach that development with more precision of what an academic coach would do in these three components, not just in reading but in helping teachers understand about parent/classroom partnership and behavior management as well. Classroom behavior management, you saw the Good Behavior Game plus some standard programs that are already going on. And the third component is parent/classroom partnering, involves a telephone, and so the parent/classroom partnering makes use of by the way a dedicated telephone that is in the classroom, it’s been tested in the Lift(?) Program that John Reed and the Oregon Prevention Research Center developed and tested a few years ago. And it consists of a teacher being trained to leave a message for the parents about what the week was all about, what’s going to happen next week, and it allows for two-way messages to go from the parents to the teacher and back again. There’s also a family classroom partnership aid, it’s a mouthful, we’re trying to get away from Title I lingo. Anyway, this is a person who typically makes $3,000 dollars and is a dedicated citizen who wants to work as a community/classroom classroom based person with the teacher and someone that the principal has trust in and can be the link to the community. So that’s vaguely the intervention.

The design consists of 20 schools, eight of which are development schools and actually Bernard Harris is one of the development school and we didn’t realize that they had an after school program going on. But in any case, and they fall into Linda Chinnia’s area, area one of Baltimore City Public Schools. The 12 schools where the randomized occurs, and by the way the eight are critically important, they’re the ones that allow us to come in and develop all the measures, do the psychometric work, try out the interventions to see where the bugs are, they were just important to the context in which the randomized field trial which is necessarily rigorous and highly structured is going on. So the 12 schools, we’ve just recently done random assignment of children and teachers and of course did random assignment earlier of which of the area schools would be the 12 and the eight. So this is the design.

This is what an analytic model looks like when you start, when you go to the work in the morning this is what you’re facing. What we’re seeing is that over on the left side you’ve got three targets that you want to improve, you want effective classroom management, you want effective academic instruction, and effective family/classroom partnership, that’s the whole day program. The mediators in all that are number one, what’s the quality of implementation that these three are done, and to do that we’re using Jack’s measures of skill attainment by the kids, which is later, improved reading skills and so on. We’re using Barbara Foreman’s measures of the quality of instruction augmented by behavioral elements. And we’re also adding to Jack’s skill measures elements of psychological well being and behavior in the interviews of each of the children. So the left side you have time sampling of quality of instruction, in the middle you’ve got the things that are supposed to happen that explain what happens on the right. And then you got, this is supposed to lead to increased achievement and of course the people that give us all the funding are over in the right waiting to see whether they’ve invested their money wisely. And so that’s what the analytic model looks like.

I want to stop at this point and let Linda talk about the school district and their side of all this negotiation. Thank you.