Workshop on Understanding and Promoting Knowledge Accumulation in Education:

Tools and Strategies for Education Research

Day 1 – June 30, 2003

Remarks by Dr. Claudia Buchmann

MS. CLAUDIA BUCHMANN: I was asked to discuss issues related to the measurement of family background, and I am very pleased to be here to do that today. As we discuss how to develop common measures of family background and socio-economic status, I think it might be useful to begin with a question, which is why is it important to measure family background well in educational research?

Well there are many good answers to that question, but let me highlight two.

The first relates to the importance of controlling for family influences, so that we can examine school effects - family background effects, and, as James Coleman stated almost 30 years ago, in the attempt to discover effects of school factors on achievement, perhaps the principal villain is the fact that student populations in different schools differ at the outset. It is not possible merely to judge the quality of a school by the achievements of the students leaving it. It is necessary to control for the variations in student input with which the teachers and staff of the school are confronted.

In other words, it is crucial to know how students in a population are distributed on a wide range of family factors which are themselves important predictors of achievement, because only then can we assess the role of the school in achieving social and economic objectives.

A second reason that we should give careful thought to how to measure family background relates to the necessity to improve our knowledge of the ways that the family, as an institution, affects children’s ability and motivation to learn and their academic achievement.

A better understanding of how family background relates to student learning can help societies formulate policies that make - to intervene in detrimental family processes or to enhance beneficial ones. Certainly, we are constantly looking for ways to alter educational institutions to improve children’s lives. This same thinking should be applied to the family institution, especially in this era of rapidly changing family life.

In many ways, the measurement of issues of family background in SES reflect the challenges of developing a common core of measures in other realms in the study of educational research, and in order to demonstrate this point, I am going to talk a little bit about how measurement of family background factors have evolved over time.

Well, the concept of family background has become increasingly complex over time. It has expanded from the initial specification of socio-economic standing, of family of origin, and this idea remains at the core of that concept today, but now it also includes measures of family structure or demographic characteristics of the family, as well as family social and cultural capital.

As I will demonstrate, for the long-standing component of SES, a common core of measures has become fairly well established, but for the other, more recent conceptualizations of family background, namely family structure and social and cultural capital of the family, there is far less consensus in terms of how to operationalize and measure these concepts.

Okay. Moving first to socio-economic status. Three components, parents’ education, parents’ occupation and family income, typically comprised the measure of family SES.

Status-attainment research begun by sociologists more than three decades ago laid the foundation for this conceptualization of socio-economic status and methodology which was usually half analysis and multiple-regression techniques to investigate the intergenerational transmission of status.

The classic studies, the American Occupational Structure by Duncan and those studies produced by Bill Suel(?) and colleagues, known as the Wisconsin Model established a framework for the measurement of family SES in educational research.

By the 1980s, more than 500 papers had attempted to replicate or extend their basic findings. Human capital models in economics in which family background and schooling decisions determined education and earnings outcomes also contributed to this growing field.

Let me summarize briefly each component of this - can you go back? - each component of socio-economic status.

Parental education is usually measured as the level or number of years of education of the parent. Since maternal and paternal education are often highly correlated, researchers usually use one or the other as a measure of parental education. In context where mothers spend more time with their children or where fathers are typically absent from the household, it is reasonable to expect that mother’s education should have a stronger impact and researchers then have used mother’s education as the - for parental education.

Another strategy has been to use the sum of both parents’ schooling as a measure of parental education.

Occupational status is typically measured via scales that have been developed to generalize the prestige associated with occupations across a wide range of societies, and much effort has gone into developing scales so that they are comparable on a global level.

Two of these scales - the Standard International Occupational Prestige Scale and the ISEI or the International Socio-Economic Index - are perhaps the most well known, and in the paper that I believe has been distributed to the committee members I discuss those scales in greater detail.

Now, developing reliable measures of family wealth or income has been more complicated than developing measures of parental education or occupational status. First, it can be difficult to get high response rates on income questions, and the accuracy of the responses to income questions are often suspect. These and other challenges with collecting income data have led many researchers to use other measures as proxies for family wealth. The example of the indices of home possession is one that I will talk a little bit more about in a moment.

Oh, in fact, just to mention that some researchers argue that these kinds of indices of home possessions are even better approximations of long-term wealth, because they can reflect over the lifetime or the purchasing power of families, while income measures only reflect the family’s income status at a particular time point.

Now, to assess the level of standardization of the measurement of socio-economic status, I looked at how several major national and international surveys over the past four decades have measured SES.

This chart - which may be hard for the people in the back of the room to see, but I’ll clarify it for you - shows how four national data sets have measured SES - Project Talent, which was conducted in the 1960s; the National Longitudinal Survey of the High School Class of 1972; High School and Beyond, which was begun it the ‘80s; and the National Educational Longitudinal Survey begun in the late 1980s and continued into the ‘90s.

And the - information that this chart is telling us is that for every survey we see measures of mother’s and father’s education, mother’s and father’s occupation and family income. So all - there is consistency here across these surveys in terms of the kinds of measures that they are including.

And also impressive is the point that all measures across these surveys can be standardized or are standardized or they could be standardized with some minor revisions across the surveys.

Looking, then to international surveys, this chart shows how socio-economic status is measured in six major international surveys in the past four decades. The first five of these were conducted by the International Association for the Evaluation of Educational Achievement, and the last one is the effort of the OECD. So the first are IEA studies. The last one, known as Pizza(?), is an ongoing effort by the OECD.

And what we see here is more standardization in some areas and less standardization in others.

First - one thing to notice is that the early surveys only ask for father’s occupation, but the more recent surveys acknowledge women’s growing labor-force participation and so they ask questions about mother’s occupation.

Pizza, the most recent survey, is most comprehensive, and measures all three components of SES. It measures father’s and mother’s education, father’s and mother’s occupation and then includes a measure of home possessions as a measure of income for wealth.

Now TIMSS, the third international - oh, can you go back? - third international math and science study, differs markedly from the studies that came before it, in that it did not include questions on parental occupation, and Larry Seuter(?), who is here today, and I were talking about the reason for this, and he informed me that the TIMSS organizers really felt that problems in gathering parental occupational status that were found in the prior IEA surveys were - questions for the TIMSS survey about how feasible it was to gather reliable and useable data on parental occupation in the TIMSS.

So in the TIMSS, we only have - the measure of SES is limited. It’s two components, parental education and home possessions.

Again, this issue of home possessions, the inclusion of home-possession questions may be useful not just for addressing the problems of inaccuracy or unreliable measures of income, but they also may be very useful when you are surveying young students about family wealth. They may be more able to answer questions about the kinds of possessions that they - have in their household than, for example, questions about their parents’ income.

Early Bulbo(?), at the University of Pennsylvania, is working to develop an index for socio-economic status using the TIMSS data on these household possessions, but he is running into a bit of a challenge, because what TIMSS did was that they allowed the countries - I failed to mention that TIMSS was a survey across 42 different countries on educational achievement, and TIMSS allowed the countries to really come up with the measures of household possessions that they were to include. So this lack of standardization across countries has created a challenge for creating a comparable measure of household possessions. So, on the one hand, while the addition of household-assets measures in TIMSS was an improvement over prior surveys, future surveys should really try to ensure at least a common for household-possession measures that may be comparable across a range of different contexts and then perhaps allow variability across countries for additional measures.

Well, these challenges notwithstanding, it is clear that one benefit of the relatively consistent measurement of family SES across both national and international surveys and a wide range of studies have resulted in accumulation of knowledge about the relationship between family SES and educational outcomes over time, and, certainly, that topic is worthy of a book, but I am going to just highlight three issues very briefly.

First, virtually all studies find that socio-economic status has a substantial impact on educational attainment and achievement across a wide range of contexts.

Second, father’s education is usually found to be a stronger determinant than occupational status or mother’s education, although these latter measures are also usually found to be important.

Finally, we generally know that family socio-economic status tends to have a larger impact on educational attainment and achievement in the earlier stages of the student’s life course than in the later ones.

Okay. Well, again, as important as families SES is, it captures only one aspect of family background. So I am going to talk a little bit about these other components.

Oh, boy. Now, is that on the 15 minutes or the 20 minutes? Okay. I’ll go very quickly through family structure.

We know that family structure also matters. Children in the presence of - in households with one parent versus two parents tend to differ in terms of their educational outcomes. There are - we also know that family size matters. The number of siblings in the household also has an impact.

But, basically, if we can go to the next slide, let me show you something interesting about the way international surveys have measured family structure, because there is an important inconsistency here.

The first and second science studies, the early studies used number of siblings and birth order to address the question of family size and number of siblings in the household. The TIMSS, though, asked a more general question about the total number of people in the household. Students were asked to indicate yes or no in terms of whether certain people lived at home with them all or most of the times, and they were asked, “Does your father live with you? Does your mother live with you? Do one or more brothers live with you?” and so on.

And the structure of this question is such that the element - the most important elements of family structure that should be most important for educational outcomes really can’t be determined. There is no way to deduce the total number of siblings from this question, the sex compositions of the siblings or the total number of adults in the household, and so I think in this case, TIMSS really demonstrates a failure of a major time and labor-consuming effort to use cumulated knowledge from past educational research and past surveys, because the result, in this case, is an unstandardized and not very useful measure of family structure. Fortunately, we see that Pizza did not replicate that mistake.

Okay. Moving on quickly to families’ social and cultural capital. Basically, these are two ideas that have become very popular among social scientists and policymakers. James Coleman, one of the earlier proponents of social capital, defined it as a social structural asset for the individual that facilities certain beneficial actions and outcomes for those who occupy a given social structure.

Within the family, basically - social capital is frequently measured as parent-child ties and the attention that parents devote to their children’s schooling. Okay?

I think I’ll skip over cultural capital and just move on to my discussion of how we measure social capital in surveys.

This - began as an example from international surveys. Again, we see a relative lack of standardization, in part, because these concepts of social and cultural capital are much more recent. They really haven’t - knowledge hasn’t really cumulated in these domains in terms of how - what precisely we mean when we talk about social and cultural capital and how precisely we should be measuring those things.

Only four of the six international surveys that I mentioned include measures of either concept, and of the three surveys that do incorporate social capital within the family in terms of their questions, these concepts are quite variously measured. So we see help with homework across all of the surveys, but then the rest of the questions that are trying to tap into these aspects of social capital are quite different across the surveys.

So here I would say that compared to research on other aspects of family background, notably socio-economic status, the study of social and cultural capital as it relates to children’s schooling is still in its early stages, and these concepts are continuing to be refined.

Now, we can talk about, in the discussion, whether standardizing these measures in some way is desirable and what we lose in that. I would suggest that at this point we would lose more than we would gain by trying to come up with really standard measures of these concepts.

Okay. To summarize quickly then, this quick look at the evolution of the measurement of family background, I think, is useful for illuminating some of the challenges in establishing uniform measures in our research, and, in particular, how we measure family background in educational research.

To the degree that we can learn from past efforts, I think it can also offer some recommendations for dealing with these challenges.

The first is that we - at least in the case of researchers who survey data, we need to recognize that researchers are at the mercy of survey designers and research can only be as good as the survey design itself. Survey designers may not always be aware of the knowledge that has cumulated in their particular field, and the design of the surveys, especially these large-scale surveys, and their instrumentation will determine the kinds of research questions that we can handle such survey data and will also determine whether specific knowledge gaps can be rectified or the kinds of answers that might be possible, and so I think this point reminds us that researchers really have a role to play in the development of large-scale surveys, both national data sets and international data sets, and greater communication between survey designers and researchers could serve to great benefit.

The second point I would like to make is that establishing - another challenge remains in establishing a fine balance - we have talked about this a little bit in the morning - between developing standardized measures on the one hand, but remaining cognizant of contextual variations and how do we remain sensitive to context and ensure that our measures can tap into contextual variation while they are also some of the comparable across studies.

One example that is interesting here comes actually from the school-effects literature that was also discussed earlier this morning, and, in particular, to compare it with the international school-effects literature, because, in response to the Coleman Report, studies by Hiaman(?) and Loxley(?) basically found that the proportion of variance and achievement that is attributable to family background was generally much smaller and that attributable to school quality was generally much larger in developing countries relative to industrialized countries, and so there was a great deal of interest in this, and by the mid-1990s more than 100 studies of school effects have been conducted in a wide range of developing countries seeking to replicate this finding.

But, importantly, some scholars have really criticized these studies, not only on the grounds that other people have mentioned, but because they used inadequate or inappropriate controls for family background, and the idea here is that if we take Western notions or imprecise socio-economic status indicators from the West and simply import them into other contexts, we will really fail to be able to see what portion of the effects that we are getting are due to mis-specification or what portion are actually due to differences in school quality versus family background effects, and so this raises the point that knowledge cumulation can also be hindered by an overly-systematic approach to measuring family background.

I think I’m out of time. So I will stop there. We can certainly talk about the huge issues of the definition of family and the household if time permits. Thank you.