completed_projects

Below is a list of the Committee on National Statistics' completed projects. Please click on the project title for a description.

Seminars

Panels

Please note that the Statement of Work for each of these completed projects is the original.

Workshops

Past CNSTAT Seminar Reports

June 18-19, 2004 – 94^th Meeting
Topic: Household Survey Nonresponse: What Do We Know? What Can We Do?
February 20, 2004 – 93^rd Meeting
Meeting – No Seminar Session
October 23-24, 2003 – 92^nd Meeting
Topic: CNSTAT leadership and DBASSE, NRC, and IOM collaboration in the KNII
May 9-10, 2003 – 91^st Meeting
Seminar Topic: The Polygraph and Lie Detection
February 21-22, 2003 – 90^th Meeting
Business Meeting – No Seminar Session
October 25-26, 2002 – 89^th Meeting
Seminar Topic: Challenges in Aligning Census Data with Public Policy Issues
May 10-11, 2002 – 88^th Meeting
Seminar Topic: Survey Automation: The Promise and the Reality
February 8-9, 2002 – 87^th Meeting
Business Meeting – No Seminar Session
October 26-27, 2001 – 86^th Meeting
Seminar Topic: Census Crossroads: The Decision on the 2000 Census Adjustment and Early Planning for 2010
May 11-12, 2001 – 85^th Meeting
Seminar Topic: Adjusting Data for Comparability Using OMB’s Race Classifications
February 9-10, 2001 – 84^th Meeting
Business Meeting – No Seminar Session
October 27-28, 2000 – 83^rd Meeting
Seminar Topic: Defining and Measuring Output in the Service Sector Economy
May 12-13, 2000 – 82^nd Meeting
Seminar Topic: Improving the Quality of Medical Care Statistics: Toward National Health Accounts
February 11-12, 2000 – 81st Meeting
Business Meeting – No Seminar Session

Panels

A Study on the Design of Nonmarket Accounts

The panel was established to examine the design of nonmarket accounts that would parallel the market-based national income and product accounts. The panel reviewed current approaches, examined data requirements and limitations, determined the priorities for developing nonmarket accounts, and suggested further research to strengthen the knowledge base about nonmarket accounting. Specific issues addressed by the panel included (1) review of efforts to develop nonmarket accounts developed by government agencies as well as by private organizations and scholars; (2) make specific recommendations on the framework and sectors for developing nonmarket accounts; (3) examine and make recommendations with respect to key data needed to develop nonmarket accounts; and (4) investigate and make recommendations for methodological research on nonmarket accounts in the areas of statistics, economics, psychology, survey research, and related disciplines.

Alternative Census Methodologies

The panel provided advice about methods to use for sampling for nonresponse follow-up and integrated coverage measurement (ICM) and the utility of administrative records for address list development and household roster development. The panel also provided advice about research leading to the development of the 2000 census design, and identified short-term and longer-term research issues for future decennial censuses, periodic surveys, and an estimates program to produce population and related demographic statistical data. The panel (1) reviewed the results of the 1995 Census Test, (2) reviewed the proposed ICM sample design and estimation procedures for the 2000 census, (3) recommended additional field tests and research to be carried out in the near term (i.e., fiscal year 1996) and as part of the 2000 Census Research, Evaluation, and Experimentation (REX) Program, (4) reviewed the evaluation of administration records in the 1995 Census Test and explored the use of administrative records in taking the decennial census in 2000.

Assess the Portfolio of the Science Resources Studies Division of NSF

The Science Resources Studies Division (SRS) of the National Science Foundation (NSF) asked the National Research Council (NRC) to undertake a review of its portfolio of data collection and analysis activities and to assist SRS in revising there activities to better meet the information needs of policy makers, managers, educators, and researchers. In response, the NRC created the Committee to Assess the Portfolio of the Science Resources Studies Division of NSF to identify gaps in NSF surveys and provide prioritized recommendations for addressing them. Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies was published in 2000.

Bureau of Transportation Statistics International Trade Traffic Study

The Bureau of Transportation Statistics (BTS) of the U.S. Department of Transportation requested that CNSTAT evaluate the accuracy and reliability of measures of ton-miles and value-miles of international trade traffic carried by highway for each State for use as formula factors for highway apportionments. CNSTAT in turn established a panel to conduct such a study and to review the findings of a BTS staff report prepared in response to PL 105-178 and its recommendations, and to explore additional data sources and measures that may be utilized for this purpose. The study resulted in recommendations for statistical quality standards that address data sources, data collection and estimation methodology.

Census Requirements in the Year 2000 and Beyond

This panel focused on the technical aspects of the study by providing scientific and technical evaluations of alternative designs for the year 2000 census and related demographic information systems. The panel identified those alternative designs proffered by the Census Bureau that merit serious investigation in the near term; evaluated research proposed by the Census Bureau on census design innovations, recommended further research projects, and evaluated the results of preliminary research by the Census Bureau and the selection of those census designs to be given further consideration; and recommended features of a census design that should be further investigated and developed for censuses in 2010 and succeeding years. This panel released three reports: Counting People in the Information Age (1994), Modernizing the U.S. Census (1994) and A Census That Mirrors America: Interim Report (1993).

Conceptual, Measurement, and Other Statistical Issues in Developing Cost-of-Living Indexes

The study investigated conceptual and measurement issues in the development of cost-of-living indices. Explicit assumptions were made underlying different approaches to index construction and the study assessed the appropriate technical use of such indices for indexing federal programs. The study also outlined a program of research and experimental measures to allow for further assessment and refinement of cost-of-living indices. At What Price?: Conceptualizing and Measuring Cost-of-Living and Price Indexes was published in 2002.

Confidential Data Access for Research Purposes

This study assessed competing approaches to promoting exploitation of the research potential of microdata—particularly linked longitudinal microdata—while preserving respondent confidentiality. The ultimate goal was for the panel to make recommendations about how microdata should optimally (from a societal standpoint) be made available to researchers. This will require, among other things, thinking about how to measure the value of the research good made possible by data production and access, as well as the risk (and associated cost) of disclosures. Such measures are needed in order to assess the tradeoff between the benefits derived from increased protection of data versus those derived from fuller data access.

The panel also focused on (1) technical, legal and statistical ingredients needed to promote arrangements within and between agencies, and also between government and private sector data producers; (2) enforcement of legal protections for data subjects and appropriate penalties for misuse, and how breakdowns in security are detected, assuming they are, and traced to responsible parties. The panel also considered the relative advantages associated with various approaches to data protection, and form recommendations about (1) alternative, less burdensome systems (e.g., internet, remote access etc.) of providing researchers with access to restricted data, and (2) cutting edge statistical techniques for manipulating data in ways that claim to preserve important statistical properties and allow for broader general data release. Expanding Access to Research Data: Reconciling Risks and Opportunities was published in 2005.

Confidentiality and Data Access

CNSTAT and the Social Science Research Council have jointly convened a panel to study issues of confidentiality and data access and to provide recommendations to federal agencies for better accommodating the increasing tension between data access and confidentiality. Support for this program was provided by NSF, the Bureau of the Census, the Bureau of Labor Statistics, the IRS Statistics of Income Division, the National Institute on Aging (NIA), the National Center for Education Statistics, and other federal agencies through contributions for the work of CNSTAT.

The panel's work builds on previous programs of the NRC and the Social Science Research Council as well as on four workshops. Three workshops addressed issues of confidentiality of and access to the Longitudinal Retirement History Survey, the Doctorate Records File and the Survey of Doctorate Recipients, and National Center for Education Statistics data. A fourth workshop focused on statistical methods of disclosure limitation and data access. The panel's report, Private Lives and Public Policies: Confidentiality and Accessibility of Government Statistics was released in late 1993. A special issue of the Journal of Official Statistics, containing papers commissioned by the panel for the March 1991 Conference on Disclosure Limitation Approaches and Data Access, was also released in 1993.

Data and Methods for Measuring the Effects of Changes in Social Welfare Programs

At the request of the DHHS Office of the Assistant Secretary for Planning and Evaluation (ASPE), CNSTAT held a conference December 12-13, 1996 on the topic of National Statistics on Health and Social Welfare Programs.

Aid to Families with Dependent Children, Medicaid, and other federal-state health and social welfare programs have changed in fundamental ways, as states have bee given increased flexibility to administer them. Further changes in legislation increased the variation in programs and the administration of them across the states. Many states have shifted the focus of welfare related programs from income support to employment. Some states have imposed time limits for benefits, some have restricted eligibility among adolescent mothers and legal aliens, and some have substituted in-kind services and vouchers for cash assistance. At the same time that welfare programs have become more varied and complex, health insurance coverage has also undergone major changes, as burgeoning managed care programs take greater responsibility for high-risk populations. As these health and social welfare programs change, so also must our national surveys on them.

Representatives of federal statistical and other agencies, along with experts from academia, gathered to discuss implications of changes in health and social welfare programs for data collection. They also identified new and anticipated needs for data, reviewed the current ways of obtaining information on participants in these programs, and suggested new approaches, including ways in which efforts can be coordinated in order to produce consistent and comparable measures and data. Providing National Statistics on Health and Social Welfare Programs in an Era of Change, a summary of the workshop, was published in 1998.

Data and Research for Policy on Illegal Drugs

Demographic and Economic Impacts of Immigration

CNSTAT and the Committee on Population (CPOP) convened this panel to assess the demographic, economic and fiscal consequences of immigration. The panel was not asked to answer questions about immigration, or to recommend policy. Rather the goal was to improve the scientific foundation for public discussion and policy making around a few issues.

DHHS Collection of Race and Ethnicity Data

CNSTAT established a panel to examine the adequacy of race and ethnicity data collected or used by the U.S. Department of Health and Human Services (DHHS) programs. The panel reviewed policies and practices, examined data requirements and limitations, and suggested improved methods. The panel consisted of 10-12 experts representing the fields of health services research, survey research, statistics, medical care, sociology, economics, private sector health insurance, and other relevant fields of research. A workshop was held December 2001 and a workshop summary was issued in spring 2003 Improving Racial and Ethnic Data on Health: Report of a Workshop. A final report Eliminating Health Disparities: Measurement and Data Needs was published in fall 2003.

Enhancing the Data Infrastructure in Support of Food and Nutrition Programs

CNSTAT convened a study to review the food consumption and nutrition data infrastructure needed to support U.S. Department of Agriculture (USDA) programs and research and to assess where data enhancements can be made to improve this infrastructure. The purpose of the study was to define the scope of current and emerging food consumption and nutrition data needs and review existing data to identify gaps and weaknesses. The study also outlined strategies for enhancing the infrastructure, such as new data collections, new sampling strategies, combining data sets, and purchasing private data sets. Strategies that increase data infrastructure efficiencies for the USDA and other agencies with related data needs were considered. The panel’s report Improving Data to Analyze Food and Nutrition Policies was published in 2005.

Estimates of Poverty for Small Geographic Areas

Evaluate Alternative Census Methods

This panel focused on the technical aspects of the study. It provided scientific and technical evaluations of alternative designs for the year 2000 census and related demographic information systems. The panel identified those alternative designs proffered by the Census Bureau that merit serious investigation in the near term; evaluated research proposed by the Census Bureau on census design innovations, recommended further research projects, and evaluated the results of preliminary research by the Census Bureau and the selection of those census designs to be given further consideration; and recommended features of a census design that should be further investigated and developed for censuses in 2010 and succeeding years.

Evaluate Microsimulation Models for Social Welfare Programs

At the request of the Department of Health and Human Services (HHS), CNSTAT established a panel to review the models used by HHS and the Food and Nutrition Service of the Department of Agriculture to evaluate proposed changes in federal welfare programs. The panel conducted a comprehensive review of these models, including (1) the role of microsimulation modeling vis-a-vis other methods of policy analysis; (2) the capabilities, theoretical and operational validity, and usability of existing models; (3) the adequacy of the model data base; (4) advances in the state of the art of computer technology, methodology for statistical data development and analysis, and methodology for expanding modeling capability; and (5) goals and priorities for model and data development over the long term.

Evaluate the Survey of Income and Program Participation

The Panel to Evaluate the Survey of Income and Program Participation (SIPP) completed a two-phase study of the survey. The Census Bureau and the Office of Management and Budget requested an independent study of SIPP five years after the survey was initiated. In the first phase, a subcommittee of CNSTAT produced a report on the goals and uses of SIPP, the quality and utility of its data products, and methodological problems that was published in August 1989.

In the second phase, the panel carried out an in-depth review of the SIPP program, examined design and content features of the survey, and considered ways to improve the uses of SIPP longitudinal data and ways to improve the nation's income statistics with data from SIPP, the March Current Population Survey, and administrative records. The panel's report, The Future of the Survey of Income and Program Participation, was published in 1993.

Evaluation of the USDA’s Methodology for Estimating Eligibility and Participation for the WIC Program

Expert Review of the Statistical Procedures for the Decennial Census (Review of the 2000 Census)

Foreign Trade Statistics

CNSTAT convened a Panel on Foreign Trade Statistics to conduct a study on the adequacy of U.S. systems of collecting data on international transactions. The panel's report, Behind the Numbers: U.S. Trade in the World Economy, was published in January 1992. Included in the panel's report are a new framework to elucidate U.S. international economic activities and specific steps for federal agencies to take to improve the accuracy, coverage, and usefulness of national statistics on merchandise trade and international services transactions. In addition, the report covers special analyses on the underreporting of U.S. merchandise exports, the substantial understatement of services imports and exports, alternative methods of data collection and dissemination, the need for increased automation, and current and future data needs.

Formula Allocation

CNSTAT conducted a panel study of statistical issues in the allocation of federal and state program funds to states and localities. The study considered the statistical estimates used as inputs to formulas, data and methods for estimating these inputs, the features of the formulas, and how estimates and formula features interact in ways that affect program goals. The study was conducted in two phases. The first phase focused on issues of interest to the U.S. Department of Education; the second phase broadened the focus. Two reports were written by this panel, Choosing the Right Formula: Initial Report (2001) and Statistical Issues in Allocating Funds by Formula (2003).

Improving Measures of Access to Equal Educational Opportunity; as Evaluation of E&S Survey Data (joint project with CFE)

Institutional Review Boards, Surveys, and Social Science Research

CNSTAT, collaborating with BBCSS and coordinating with the IOM, organized a panel for a study on Institutional Review Boards, Surveys, and Social and Behavioral Science Research. The panel reviewed proposed methods of human subjects' protection in social science data collection. It focused on the structure, function, and performance of the institutional review board system put in place by DHHS largely to guide clinical and biomedical research that also affects social science researchers. The final report will be helpful to all federal agencies engaged in social science research and data collection, to academic social scientists, and to those who organize and serve on Institutional Review Boards that review social and behavioral research. The report will recommend ways of assuring the protection of human subjects in social science research that can accommodate common differences between social/behavioral and clinical/biomedical research. Two publications of this panel were Letter Report: Protecting Participants in Behavioral and Social Science Research (2002) and the final report Protecting Participants and Facilitating Social and Behavioral Sciences Research (2003).

_____________________________________________________________________________

Integrated Environmental and Economic Accounting

CNSTAT convened a panel study to examine the objectivity, methodology, and applicability of integrated environmental and economic accounting in the context of broadening the national economic accounts. The panel reviewed the approaches used by the Bureau of Economic Analysis (BEA) and others to determine the value of environmental resources, recommend improvements, and suggested further research that would strengthen the knowledge base concerning such valuation. The panel considered the objectives or goals of such broadening, and compared the methods employed by BEA with methods engaged by other countries, methods suggested by research performed in other countries statistical agencies, and methods developed by non-governmental research. The panel advised BEA on the strengths and weaknesses of different approaches, recommended improvements to BEA’s methods, and recommended areas for further research.

The panels review was conducted in four parts. First, it examined the recently developed extensions that broaden the national accounts and examined progress made by other national statistical agencies in the introduction of augmented accounts concerning the environmental area. Second, the panel reviewed the first phase of BEA’s augmented environmental accounts, which included revisions incorporating the reduction, through production, of subsoil assets such as oil and gas. Third, it reviewed BEA’s proposed plans and methodology for the creation of augmented accounts that extend coverage to renewable, appropriable resources, e.g., water and timber. Lastly, it reviewed BEA’s proposed plans to extend the augmentation to environmental resources such as clean air.

International Capital Transactions

The panel was convened as the second phase to the Panel on Foreign Trade Statistics; its focus was on international capital flows. The panel's charge was to (1) examine the causes of the burgeoning statistical discrepancy in the U.S. balance-of-payments accounts; (2) review data collection systems on portfolio transactions and propose alternative methods; (3) assess the adequacy of data on U.S. government international capital transactions; (4) obtain the views of users on the adequacy of existing data and the views of filers on compliance requirements; (5) review the adequacy of data collection on direct investment capital flows; and (6) review current methods to estimate portfolio and direct investment incomes and recommend improvements. The final report, Following the Money: U.S. Finance in the World Economy was published in October 1995.

Issues in Developing Cost of Living Indices for Indexing Federal Programs

This study investigated conceptual and measurement issues in the development of cost-of-living indices. Explicit assumptions were made underlying different approaches to index construction and the study assessed the appropriate technical use of such indices for indexing federal programs. The study also outlined a program of research and experimental measures to allow for further assessment and refinement of cost-of-living indices. The report At What Price?: Conceptualizing and Measuring Cost-of-Living and Price Indexes was published in2002.

Methods for Assessing Discrimination

A panel of scholars considered the definition of racial discrimination, assessed current methodologies for measuring it, identified new approaches, and made recommendations about the best, broad methodological approaches. This panel gave the policy and scholarly communities new tools for assessing the extent to which discrimination continues to undermine the achievement of equal opportunity by suggesting additional means for measuring discrimination that can be applied not only to the racial question but in other important social arenas as well. The panel conducted a thorough evaluation of current methodologies for measuring discrimination in a wide range of circumstances where it may occur.

Finally, the panel considered how analyses of data from other sources could contribute to findings from research experimentation, such as the HUD paired tests. The panel’s report Measuring Racial Discrimination (2004) recommended further research as well as the development of data to complement research studies.

_____________________________________________________________________________

National Health Care Survey

At the request of the National Center for Health Statistics, CNSTAT and the Institute of Medicine's Board (IOM) on Health Care Services established a panel to evaluate plans for a National Health Care Survey and to recommend improvements.

The National Center for Health Statistics plans a major revision and expansion of its data collection systems into a new integrated National Health Care Survey. The Panel on the National Health Care Survey identified the principal current and future needs for health care data by public and private health policymakers, health care providers, health services researchers, and others and evaluated statistical aspects of the proposed survey and the extent to which the survey can meet the identified needs for data. The panel's report Toward a National Health Care Survey: A Data System for the 21st Century was published in February 1992.

Operational Test Design and Evaluation of the Interim Armored Vehicle

Panel Study of Data and Methods for Measuring the Effects of Changes in Social Welfare Programs

CNSTAT convened a panel of experts to review data needs and methods for evaluating the outcomes of recently-enacted, far-reaching changes in social welfare programs. Of particular interest were tracking and assessing the effects of program changes on people who leave the welfare rolls. The panel considered alternative federal and state data sources; appropriate research methods for analysis and inference, considering the limitations of available data; and findings from research and evaluation of previous welfare program changes. The panel’s reports included: Evaluating Welfare Reform : A Framework and Review of Current Work, Interim Report (1999) and Evaluating Welfare in an Era of Transition (2001) and the panel issued a final report Studies of Welfare Populations: Data Collection and Research Issues (2001) at the conclusion of the study with recommendations for data and analysis methods.

Performance Measures and Data for Public Health Performance Partnership Grants (PPGs)

CNSTAT convened a panel study to review performance measures for state and federal government use in assessing progress toward objectives on public health Performance Partnership Grants (PPGs) in six areas: mental health, substance abuse, HIV/STD/TB, chronic diseases and prevention of disabilities, immunizations, and preventive health and health services. The review (1) identified those results that states and other interested parties want to achieve through the PPGs that can be stated as measurable objectives and measured at the state and national level either now or with small modifications to data systems; (2) identified objectives relevant to the PPGs that are not being measured now but that are important to states and the federal government and recommend other improvements to the development of performance measures for PPGs; and (3) recommended improvements to state and federal surveys and data systems to facilitate future measurement of these objectives.

The panel commissioned Emerson Elliott and John H. Ralph to prepare a paper entitled, Quality Education Data: Unprecedented Opportunity for a Decade to Build. The paper was helpful in informing the panel about experiences with education data quality improvements of relevance to the panel's deliberations on public health performance measurement.

Poverty and Family Assistance

CNSTAT convened a panel to study statistical issues involved in measuring and understanding poverty and in establishing a national minimum welfare benefit for low-income families with children. The study was not to set a new poverty threshold or to evaluate a proposed program of welfare benefits. Rather, it concentrated on the concepts, information, and measurement methods needed for such purposes. Topics the panel considered include (1) the conceptual soundness of alternative proposed measures of need and of resources to meet those needs; (2) the suitability of alternative concepts and recipient units and periods of accounting for the various purposes for which information about poverty is desired; and (3) the current and potential availability of the data required to implement different concepts and ways of improving the quality of available data. Its report, Measuring Poverty: A New Approach, was released in April 1995.

Principles and Practices for a Federal Statistical Agency

CNSTAT has at times been asked for advice on principles and practices that make for an effective federal statistical agency. The committee has prepared a report that includes discussion of aspects of independence of a federal statistical agency, the roles of research and analysis in a statistical agency, policies of confidentiality, quality assurance programs, professional advancement of staff, dissemination activities, cooperation with data users and providers, and coordination with other statistical agencies. Principles and Practices for a Federal Statistical Agency, the first report, was published in October 1992. A second edition Principles and Practices for a Federal Statistical Agency was published in 2001.

Quality Improvement in Student Financial Aid Programs

At the request of the Department of Education, a panel was established to conduct a study of quality improvement in student financial aid programs operated by the Department of Education under Title IV of the Higher Education Act. The Panel on Quality Improvement in Student Financial Aid Programs assessed the effectiveness of the current quality control system for student financial aid programs and, where evidence warranted, proposed new procedures for more effectively and efficiently providing financial aid to students. Among special topics the panel considered were (1) the methodological and statistical integrity of current quality control studies; (2) whether information collected effectively guides strategies to reduce errors; (3) knowledge and experience gained from practices in other federal assistance programs and in service industries; and (4) the applicability of different models of quality control for federal programs, in particular the use of financial incentives and rewards and alternative approaches to institutional accountability. The panel's report, Quality in Student Financial Aid Programs: A New Approach was released in August 1993.

Research and Development Statistics at the National Science Foundation

A committee operating under the auspices of CNSTAT conducted an in-depth and broad-based study to look at the NSF’s Science Resources Statistics (SRS) Research and Development Statistics Program. The goal was to look at how R&D surveys are conducted and how they should be conducted to capture the country's R&D activities over the coming decade.

The CNSTAT committee conducted its work in cooperation with a separately appointed panel of the Board on Science, Technology, and Economic Policy (STEP). The STEP panel planned and conducted a workshop and wrote a report focusing on the issues of composition, structure, sourcing and location, particularly in the context of the industrial R&D and federal funds surveys, covering the majority of U.S. R&D funding and performance. The STEP panel's report informed the deliberations of the CNSTAT committee

The CNSTAT committee reviewed existing R&D data collection systems and relevant literature, commission appropriate papers, attended and participated as appropriate in the STEP panel workshop; identified gaps in current methodology, integrated the report of the STEP panel with the committee's own extended findings, and held a separate workshop on R&D measurement methodology before preparing its report. Measuring Research and Development Expenditures in the U.S. Economy: Interim Report was published in 2004. Measuring Research and Development Expenditures in the U.S. Economy was published in 2004 and Research Data and Development Needs: Proceedings of a Workshop was published in 2005.

Research Agenda and New Data for an Aging World (with Committee on Population)

The panel was charged with (1) Identifying the scientific opportunities for conducting policy-relevant comparative research on aging; (2) reviewing sources of socioeconomic, demographic, and health data about aging populations and analyzing the strengths and weaknesses of current international data collection efforts for addressing various countries' national priorities in aging research; (3) outlining the data collection efforts, that if done internationally, would advance our understanding of the aging process around the world; and (4) identifying potential obstacles to comparative work on aging such as national variations of common definitions or differences in laws governing privacy and access to data among different countries. Preparing for an Aging World: The Case for Cross-National Research was published in 2001.

Residence Rules in the Decennial Census

The panel was charged with examining census residence rule issues and making recommendations for research and testing to develop the most important residence rules for the 2010 census. Recommendations address potential ways to modify census residence rules to facilitate more accurate counting of the population or identify the reasons why the rules should stay the same. The panel considered residence rules in terms of how they contribute to or inhibit an accurate count of the population.

Retirement Income Modeling

CNSTAT convened a panel to carry out a two-phase study on retirement income modeling. This panel was charged with providing guidance to the Pension and Welfare Benefits Administration (Department of Labor) and other relevant agencies (Social Security Administration [SSA], Treasury Department, Pension Benefit Guaranty Corporation, and NIA on needed improvements in simulation models, data sources, and behavioral research to support analysis of alternative retirement income policies.

In Phase One, the panel identified policy issues and assessed the strengths and weaknesses of existing models, data, and research, issuing an interim report in July 1995 to indicate priority areas for attention. In phase two, the panel will consider in greater depth the kinds of models and data that are needed and provide a framework for research and development. A workshop was held in September 1994 in conjunction with Phase One of the study. Papers from the workshop on the state of research knowledge in such areas as savings behavior, labor supply and demand, and health status and health care costs were compiled into the report, Assessing Knowledge of Retirement Behavior, which was published in early 1996. The panel issued a final report, Assessing Policies for Retirement Income: Needs for Data, Research, and Models in 1997.

Review Evaluation Studies of Bilingual Education

At the request of the Department of Education, the Panel to Review Evaluation Studies of Bilingual Education reviewed and assessed the methodology of data collection and analysis of two major studies of bilingual education programs. The studies are a national longitudinal study of the effectiveness of instruction of limited-English-proficient students and a study to compare the effectiveness of three different instructional strategies for bilingual education for such students.

The panel reviewed the methods of data collection and analysis for potential sources of error and assessed implications for some of the principal findings. The panel also assessed whether additional analyses of the data from either study would strengthen or broaden the findings and explored alternative ways to compare the different instructional strategies in order to provide advice to the Department of Education on commissioning and managing similar evaluation studies in the future. The panel's report, Assessing Evaluation Studies: The Case of Bilingual Education Strategies, was published in June 1992.

Review of the Scientists and Engineers Statistical Database System (SESTAT) 2000 Decade Design

CNSTAT conducted a one-day workshop guided by an appointed workshop steering committee to bring together the NSF’s Division of Science Resource Studies, academic, and other experts to examine and discuss the advantages, disadvantages, costs, analytic implications, etc. of three alternative design approaches for the SESTAT under consideration by NSF. To accomplish this study, CNSTAT appointed a panel of four or five experts representing the fields of applied sampling, statistics, and survey research. The committee produced a brief report Improving the Design of the Scientists and Engineers Statistical Database System (SESTAT) consisting of an executive summary of the workshop proceedings and recommendations of the committee.

Review of the Social Security Administration’s Disability Decision Process Research

An expert committee of 14 members was developed to conduct a study to review the SSA’s disability decision process research including the Disability Evaluation Study (DES) and to make recommendations. The study committee reviewed the scope of work for the request for proposals for the DES and provided ongoing advice on approach and scientific methods as SSA developed the new decision process and conducted relevant data collection activities in the DES. The committee activities included (but were not limited to) reviewing the preliminary design of the DES, reviewing the research plan and timeline for developing a new decision methodology, assessing the results and findings of the research undertaken and planned, and offering advice on the consequences of alternative disability determination processes. The panel’s report, The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs, was published in 2002.

Review of the Statistical Program and Practices of the Bureau of Transportation Statistics

CNSTAT and the Transportation Research Board convened a panel study to review the statistical program of the Bureau of Transportation Statistics and its practices to improve the quality of transportation statistics throughout the Department of Transportation. The panel study focused on the statistical policy function of the agency and its relationships to other Department of Transportation agencies, other federal statistical agencies, and other data providers and users, including state agencies and the private sector. The panel's report, The Bureau of Transportation Statistics: Priorities for the Future was published in 1997.

Review of the U.S. Department of Agriculture’s Measurement of Food and Hunger

USDA requested CNSTAT to convene a panel of experts to undertake a two-year study in two phases to review the concepts and methodology for measuring food insecurity and hunger and the uses of the measures. The charge specified that during Phase 1 of the study a workshop would be held to address the key issues laid out for the study and a short report was prepared based on the workshop discussions and preliminary deliberations of the panel. Measuring Food Insecurity and Hunger: Phase 1 Report presented the panel’s preliminary assessments of the food security concepts and definitions; the appropriateness of identifying hunger as a severe range of food insecurity in such a survey-based measurement method; questions for measuring these concepts; and the appropriateness of a household survey for regularly monitoring food security in the U.S. population. It provided interim guidance for the continued production of the food security estimates In Phase 2 of the study, the panel considered in more depth the issues identified in Phase 1 relating to the concepts and methods used to measure food security and made recommendations as appropriate. The final report Food Insecurity and Hunger in the United States: An Assessment of the Measure published in 2006 primarily focused on the Phase 2 charge.

In July 2004, CNSTAT convened the Workshop on the Measurement of Food Insecurity and Hunger. The four background papers that were prepared and subsequently presented at the workshop are below.

In Phase 2 of this project, the panel commissioned four additional background papers from experts in the areas of hunger, methods, cognitive aspects in questionnaire development, and comparison of selected surveys to provide expert and detailed analysis of some of the key issues beyond the time and resources of its members. These can be accessed below.

Risk and Prevalence of Elder Abuse and Neglect

The Panel to Review Risk and Prevalence of Elder Abuse and Neglect focused on assessing the knowledge base for understanding the nature of elder abuse e.g., what constitutes elder abuse and neglect, whether or not elder abuse is event based or person based, the definition of elder abuse, and the risk factors associated therewith. They addressed the current status of national, state, and local efforts to assess the incidence and prevalence of elder abuse and focused on questions such as which data systems currently measure elder abuse and how? In addition, the panel addressed the many methodological and statistical issues, such as the measurement of relatively rare events that people are reluctant to report, how to combine information from different sources, and the importance of region-specific estimates, and factors involved in prevention of elder abuse and in intervention. The report Elder Mistreatment: Abuse, Neglect, and Exploitation in an Aging America was published in 2003.

State Children’s Health Insurance Program Oversight Committee

As an outgrowth of prior requests, CNSTAT, in collaboration with the Board on Children and Families, convened a planning workshop to examine the adequacy of the federal statistical system to inform efforts to design, implement, and evaluate policies aimed at improving the health and well-being of children and families. The workshop held March 31-April 1, 1994 convened a diverse group of interested parties, including representatives from the federal research and statistical agencies, researchers and policy analysts, and executive branch and legislative staff, for the purpose of identifying the most pressing data needs in the area of child and family policy, the capacity of existing national data to inform these needs, and potential strategies for rectifying identified shortcomings. A report summarizing the workshop, including publication of papers commissioned for the workshop, Integrating Federal Statistics on Children, was published in September 1995.

Statistical Methods for Testing and Evaluating Defense Systems

CNSTAT convened a panel to investigate statistical methods to improve effectiveness and efficiency of testing and evaluating weapon systems as part of the defense acquisition process. Through critical reviews of case studies and working together with those in the Department of Defense involved in weapon system testing and evaluation, the panel was able to (1) gain a deeper understanding of testing in the acquisition process; (2) identify where and assess how alternative statistical methods could improve decision making; and (3) recommend specific methods and ways in which they can be incorporated to improve operational testing and evaluation. The panel's scope of activities included developing measures of effectiveness, designing operational tests and experiments with guidelines for determining the amount and nature of testing required, developing tests and models that incorporate information from previous stages in the acquisition process (recommendations on developmental testing would also be included), and representing and characterizing all uncertainties in presenting results.

Study of the Social Science Research Program in the Economic Research Service (ERS)

CNSTAT convened a panel to assess the quality and effectiveness of the social science research program, including the management and structure of ERS research. This agency provides economic and other social science information and analysis for public and private decision making relating to agriculture, food, natural resources, and rural development. ERS research falls into five areas: analysis of domestic and international markets and prospects for American agriculture; food quality; nutrition and safety; rural environmental issues; rural economic development. The panel issued the following report: Sowing Seeds of Change: Informing Public Policy in the Economic Research Service of USDA.

Study to Review the Scientific Evidence on the Polygraph

CNSTAT and the Board on Behavioral, Cognitive, and Sensory Sciences under the NRC’s Commission on Behavioral and Social Sciences, proposed to convene a panel of experts to review the scientific literature on polygraph tests, including what is known about the effect of medications, sleep deprivation, and illnesses on the physiological responses measured, and assess its use for personnel security screening. Polygraph tests are used by a number of federal agencies to help determine employment qualifications. The test involves measuring the physiological responses of an individual while he or she is asked both general and specific questions. The variability in the test results has made them generally inadmissible in a court of law. The panel reviewed other techniques that may be adapted to similar purposes in order to provide a comparative evaluation of the polygraph and to suggest directions for future research.

Utilization of Women-Owned Small Businesses in Federal Contracting

A workshop was planned and conducted to review and discuss the methodology for the U.S. Small Business Administration (SBA) to use to analyze information on the use of women-owned small businesses (WOSB) in federal contracting. The workshop discussion topics included the accuracy, soundness, and reliability of data, analytical methods, and other methodology used to ascertain the utilization of WOSBs in federal contracting. Participants discussed the definition of "underrepresentation" used in an SBA draft report and its potential usefulness in judicial proceedings. Furthermore, workshop participants discussed appropriate regression methods to investigate correlates of gender discrimination, the potential usefulness of additional variables in the analysis (e.g., firm size, bonding availability, etc.), the appropriate study sample size and recommended sample sizes for further or extended studies, the appropriate amount of historical data for analysis, and methods and analyses that could help SBA explain any identified industry-specific disparities and be of use to SBA in determining any remedial action. A report based on the workshop was published in 2005 Analyzing Information on Women-Owned Small Businesses in Federal Contracting.

Workshops

Demographic, Economic, and Social Effects of Immigration

The President's Commission on Immigration Reform requested the assistance of CNSTAT and CPOP in convening a small group of impartial experts to review studies on the impact of immigration, particularly illegal immigration, on local fiscal costs. CIR is a congressionally mandated commission, chaired by Barbara Jordan that looked at the overall questions of immigration. A focus of this workshop was on critically reviewing the methodologies that have been employed for measuring such costs. This workshop was held in September 1994. A report, Local Fiscal Effects of Illegal Immigration: Report of a Workshop, was published in 1996.

Disability Statistics

CNSTAT developed a panel study to review the systems of data collection on disability, to identify strengths and weaknesses, and to recommend improvements in definitions and in methodology and content of surveys. A special focus was on data useful to formulate, implement, and evaluate disability policies and programs. In preparation for this study, two workshops were held—the first, a Workshop on Forecasting Disability, was held in October 1993. Participants reviewed the data and methods used in determining trends in disability and discussed implications and possible explanations. A report of this workshop was released in September 1994.

Federal Standards for Race and Ethnicity Classification

The Office of Management and Budget requested that CNSTAT convene a workshop to discuss a possible revision of the standard federal policy governing data on race and ethnicity by identifying issues and problems with race and ethnic classification. A workshop was held February 17-18, 1994 and reflected the possibly differing views on what is measured from the social sciences, statistics, behavioral sciences, and civil rights legal perspectives, and to incorporate the views of representatives of ethnic groups and other users of the data. The workshop reviewed the classification system and how it has evolved and determine some of the problems of the system in meeting the needs for public policy; for legislation concerning the distribution of federal funds, anti-discrimination, and equal opportunity; for Congressional redistricting; and for statistical and research purposes. A report, Spotlight on Heterogeneity: The Federal Standards for Racial and Ethnic Classification, Summary of a Workshop, was published in January 1996.

Forecasting Survival, Health, and Disability

CNSTAT cosponsored a workshop to consider patterns of mortality, morbidity, and disability among older people in the United States. The workshop, which was cosponsored by IOM and the NRC's CPOP, was held in March 1992. Workshop participants compared, contrasted, and critiqued the models and assumptions used by various researchers to project levels of mortality, morbidity, and disability in the next few decades. A report of the workshop was published Forecasting Survival, Health, and Disability—Summary of a March 1993 Workshop. _____________________________________________________________________________

Improving Economic Statistics for the U.S. National Accounts

A workshop focusing on measurement and presentation of the nonprofit sector was convened by the CNSTAT in May 1992. This was the first workshop in a proposed series to address specific topics related to improving source data for the national economic accounts as the U.S. seeks to bring the accounts into conformity with the U.N. System of National Accounts. A second workshop on measurement and presentation of the government sector was held in November 1993. The report of the first workshop was published in a special issue of Voluntas, the international journal of voluntary and nonprofit organizations, in February 1994. The report of the second workshop, Measuring the Government Sector of the U.S. Economic Accounts, was published in 1996.

Improving Theory and Research on Quality Enhancement in Organizations

At the request of NSF, CNSTAT hosted a workshop September 5-7, 1996 on improving theory and research on quality enhancement in organizations. The workshop was co-chaired by Richard Scott, Department of Sociology, Stanford University, and Robert Cole, Haas School of Business, University of California, Berkeley.

Research funded by NSF in its Transformations to Quality Organizations (TQO) program involved a broad range of disciplines with only modest depth in any given substantive area. Advisors suggested that future research should forge stronger links between relevant social science theory and organizational quality principles and objectives. The workshop provided an opportunity to bring together researchers from a range of disciplinary perspectives (e.g., sociology, anthropology, management, engineering, and operations research, organizational research, and statistics) to discuss several areas related to quality improvement in organizations. These areas included quality principles for organizations and conceptualizing and measuring organizational performance. Other areas such as organizational theory, organizational training, and tacit knowledge in organization were introduced and warrant further discussion. The workshop was to be a first step in the development of unifying frameworks for future research that would emphasize connections between quality-related organizational research and relevant social science theory. Participants identified many promising lines of inquiry for cross-disciplinary research. A brief report of the workshop discussions, Improving Theory and Research on Quality Enhancement in Organizations: Report of a Workshop, was released in the fall of 1997.

Priorities for Data on the Aging Population

At the request of the NIA, CNSTAT, in collaboration with CPOP, convened a Workshop on Priorities for Data on the Aging Population. The workshop, held at the National Academy of Sciences on March 4-5, 1996, was chaired by Dorothy Rice, Institute for Health and Aging, University of California, San Francisco.

The workshop provided an opportunity to identify how the population at older ages in the next few decades will differ from the older population today, to understand the underlying causes for those changes, to anticipate future problems and policy issues, and to suggest how future needs for data for research; for understanding the important social, economic, and health conditions of the older population; and for informing public policy can be met. Discussions of the future were followed by an assessment of how current major data collection activities could serve to meet some of these needs. Participants were also asked to identify, for a few selected issues, the more important data gaps that would remain and, after taking stock of the future environment for data collection, to suggest approaches to obtaining the needed data.

Participants included members of CNSTAT and CPOP, investigators for the major national surveys on the aging population, representatives of relevant federal agencies, and other invited experts. A report of the workshop discussions, Improving Data on America's Aging Population: Summary of a Workshop, was published in 1996.

Statistical Issues in Defense Analysis and Testing

At the request of the Department of Defense, CNSTAT convened a workshop on modeling and simulation for operational testing of weapons systems. The workshop brought together defense analysts, who presented and discussed areas in which they are seeking improvements through statistical applications, and statisticians and other experts, who suggested relevant methods and approaches. The two-day workshop was held in September 1992. A report of the workshop was issued Statistical Issues in Defense Analysis and Testing: Summary of a Workshop (1994).

Statistical Methods in Software Engineering for Defense Systems (joint project with CATS)

U.S. Immigration Statistics

With the sponsorship of the Immigration and Naturalization Service and the National Institute of Child Health and Human Development, CNSTAT and CPOP hosted a two-day workshop in September 1992 to assess the needs for data collection on the topics of immigration and immigrant adjustment. The workshop discussed how immigration data could be improved to support policy analysis and development, especially for measuring the impact of immigration on U.S. society. Participants also discussed what data are requir