The National Academies: Advisers to the Nation on Science, Engineering, and Medicine
NATIONAL ACADEMY OF SCIENCES NATIONAL ACADEMY OF ENGINEERING INSTITUTE OF MEDICINE NATIONAL RESEARCH COUNCIL
Current Operating Status

The National Academies Testimony before Congress

Public Laws Containing Studies for the National Academies

Briefings to Congress

Congressionally Mandated Reports

Policy Statements and Historical Documents

The OCGA staff

Request a Report (Congressional and Government Staff Only)


Mailing Address:
The Office of Congressional and Government Affairs
The Keck Center of the National Academies
Keck WS1008
500 Fifth Street, NW
Washington, DC 20001
Tel: (202) 334-1601
Fax: (202) 334-2419

Back to Main Page

Date:03/16/2000
Session:106th Congress (Second Session)
Witness(es):Thomas S. Hartwick
Credentials:  TRW (retired) and Chairman, Committee on Assessment of Technologies Deployed to Improve Aviation Security, National Materials Advisory Board, Commission on Engineering and Technical Systems, NRC/NAS
Chamber:House
Committee:Aviation Subcommittee, Committee on Transportation and Infrastructure, U.S. House of Representatives
Subject:Aviation Security (Focusing on Training & Retention of Screeners)

UPDATE ON AVIATION SECURITY

Statement of
Thomas S. Hartwick, Ph.D.
10510 196th St. SE
Snohomish, WA 98296
360/668-4257
Chairman of the Committee on Assessment of Technologies Deployed to Improve Aviation Security
National Research Council/National Academy of Sciences

before the
Subcommittee on Aviation
Committee on Transportation and Infrastructure
U.S. House of Representatives

March 16, 2000

Mr. Chairman and Members of the Subcommittee:

My name is Dr. Thomas S. Hartwick. I am the chairman of the National Research Council (NRC) Committee on Assessment of Technologies Deployed to Improve Aviation Security. As you know, this study is being conducted in response to a Congressional directive (Section 303 PL 104-264, 1996) that the Federal Aviation Administration (FAA) engage the NRC to study the deployment of airport security equipment. The NRC is the operating arm of the National Academy of Sciences, National Academy of Engineering, and the Institute of Medicine, chartered by Congress in 1863 to advise the government on matters of science and technology. The FAA requested that the NRC assess the operational performance of explosives-detection equipment and hardened unit loading devices (HULDs) in airports and compare it to performance in laboratory testing to determine how to deploy this equipment more effectively to improve aviation security. As requested by Congress, this study addresses the following issues:

1. Assess the weapons and explosive-detection technologies available at the time of the study that are capable of being effectively deployed in commercial aviation.

2. Determine how the technologies referred to in paragraph (1) could be used more effectively to promote and improve security at airport and aviation facilities and other secured areas.

3. Assess the cost and advisability of requiring hardened cargo containers to enhance aviation security and reduce the required sensitivity of bomb-detection equipment.

4. On the basis of the assessments and determinations made under paragraphs (1), (2), and (3), identify the most promising technologies for improving the efficiency and cost-effectiveness of weapons and explosive detection.

In my opinion, it is important to note that there are several "threat vectors" or paths by which a terrorist could attack an airplane. These are presented in Figure 1. In my testimony this morning I will only focus on security measures deployed by the FAA for detecting and containing explosives introduced by checked baggage and carry-on baggage.

In November 1999 the NRC Committee on Assessment of Technologies Deployed to Improve Aviation Security (the Committee) published the report, "Assessment of Technologies Deployed to Improve Aviation Security: First Report" (first report)1. This report is the first of four planned reports assessing the deployment of technologies (i.e., equipment and procedures) by the FAA. In the first report the 1997-1998 deployment of technologies by the FAA to improve aviation security was assessed. This report, which was submitted to you in advance of this hearing is the basis for my testimony today (March 16, 2000).

DEPLOYED TECHNOLOGIES

In 1997, the FAA was directed by President Clinton and authorized by Congress (PL104-264, PL 104-208) to deploy 54 FAA-certified explosives-detection systems2 (EDS) and more than 400 trace explosives-detection devices (TEDDs) at airports around the country. The purpose of these deployments was to prevent attacks against civil aviation. The congressionally mandated deployment of bulk explosives-detection equipment began in January 1997 and continued throughout 1998. The FAA formed the Security Equipment Integrated Product Team (SEIPT) to carry out this deployment. The SEIPT assessed the availability of explosives-detection equipment

FIGURE 1 Threat vectors. The paths by which people, baggage, and equipment board a plane are also routes by which threats may board a plane. capable of being effectively deployed in commercial aviation and formulated a plan to deploy this equipment in airports throughout the United States. In a separate program, the FAA has tested HULDs designed to contain a discrete explosive blast.

The FAA's aviation security equipment and procedures include bulk3 explosives-detection equipment, TEDDs, HULDs, computer-aided passenger screening (CAPS), and positive passenger-bag matching (PPBM). These equipment and procedures are described in the first report.

FINDINGS

It is well documented4,5 that the FAA/SEIPT is behind schedule in the deployment of aviation-security equipment. In 1997, Congress provided $144.2 million for the purchase of commercially available screening equipment, and the FAA/SEIPT planned to deploy 54 certified EDSs and 489 TEDDs by December 1997. In addition, the FAA planned to implement CAPS fully by December 1997. Once it became apparent that these goals could not be met, the FAA set a new goal of deploying 54 certified EDSs, 22 noncertified bulk explosives-detection devices, and 489 TEDDs by December 31, 1998. The FAA also planned to implement CAPS fully by December 31, 1998. As of February 28, 2000 93 certified EDSs, 18 noncertified bulk explosives-detection devices, and 552 TEDDs had been installed in airports, and CAPS and PPBM had been adopted by all major U.S. carriers. In addition, 14 HULDs have been deployed to three airlines for operational testing.

The committee concluded that the combined efforts of the government, the airlines, and the airports to date have been effective in deploying aviation security technologies (improving aviation security to a level that will be quantified when additional data are collected during future studies), although, because of the urgent need for immediate action against incipient terrorism, equipment and procedures were implemented rapidly without regard for how they would contribute to a total architecture for aviation security (TAAS). The committee believes that definition of such an architecture is essential to the success of this program, hence, it suggests formality in defining and using a TAAS. That is, although the capacity of individual pieces of equipment to discretely improve security at the point of deployment is known to some degree, the integrated effect of the total deployment of equipment and the implementation of procedures on the whole of aviation security is not. After much deliberation, the committee concluded that the performance of the TAAS could be measured by a single factor, the security enhancement factor (SEF), which will enable a quantitative evaluation of the performance of diverse deployment scenarios and show the importance of specific elements (e.g., explosives-detection equipment) to the performance of the TAAS.

RESPONSE TO CONGRESS

Protecting civil aviation against terrorist threats is a complex problem. Given the short response time and the complexity of the terrorist threat, the committee concluded that the research and development and deployment by the FAA and others has been successful in qualitative terms. The urgent need for security equipment and procedures, expressed by the White House Commission on Aviation Security and Safety6 and by Congress in 1997, did not leave time for extensive system analyses. Therefore, the FAA proceeded with the deployment of hardware as it became available. The absence of a system architecture is the basis for the major recommendations of the committee. Nevertheless, the FAA will have to address these issues in the future.

Explosives-detection equipment and HULDs are part of a total system architecture and should be evaluated in the context of a TAAS. Although the FAA, its contractors, the airlines, and the airports have adopted some elements of the total systems approach, in the committee's opinion they have not gone far enough. This study, and future aviation security studies conducted by the NRC, will be most useful to the FAAif they adopt the recommended comprehensive TAAS approach. Furthermore, adopting the TAAS approach will enable the FAA (and others) to characterize improvements in aviation security quantitatively using the SEF.

The committee has addressed (in part) the four points raised by Congress below. For clarity these points are listed again, followed by the relevant conclusions and recommendations.

1.Assess the weapons and explosive-detection technologies available atthe time of the study that are capable of being effectively deployed in commercial aviation.

To date this study has focused on explosives-detection technologies. While it is conceivable that some of these technologies could also be used for weapons detection, this topic was not addressed in the first report

Bulk Explosives-Detection Equipment

The vast majority of bulk explosives-detection equipment deployed is the FAA-certified In Vision CTX-series EDS (explosives-detection system). Most of the performance data on this equipment was generated during laboratory testing –largely certification testing- at the FAA Technical Center. However, in terms of evaluating the detection rates of the equipment, certification tests only reflect the ability of the equipment to detect a bag that contains an explosive, and the detection rates are based on bag-alarm rates. That is, an explosive is considered to be detected if the alarm is set off for the bag containing the explosive, even if the alarm is triggered by a nonexplosive object in the bag. Certification testing does not measure alarm resolution and does not include testing in the operational environment of an airport, making it difficult to assess explosives-detection technologies for deployment. In the committee’s opinion, some of the unanticipated problems encountered with the CTX-5000 SP in the field can be reasonably related to the limitations of certification testing. Undercurrent certification guidelines, equipment certified in the future may encounter similar problems.

Recommendation. During certification testing, the FAA should, whenever possible, measure both true detection rates (i.e., correctly identifying where the explosive is when an alarm occurs), and false-detection rates (i.e., an alarm triggered by something other than the explosive in a bag that contains an explosive). The FAA should also include the ability of explosives-detection equipment to assist operators in resolving alarms (including in an airport) as part of certification testing. Alarm resolution should be included in the measurement of throughput rate, detection rate, and false alarm rate.

Trace Explosives-Detection Devices

TEDDs are widely used in airports, but no comprehensive methodology has been developed to evaluate their effectiveness, such as standard test articles or instrument and operator requirements. Because no standard test articles for TEDDs have been demonstrated—and because of the resultant inability to separate instrument and operator performance—it is not possible to measure the performance of TEDDs.

Recommendation. The FAA should develop and implement a program to evaluate the effectiveness of deployed trace explosives-detection devices. This evaluation should include measurements of instrument and operator performance, including measurements in the deployed (i.e., airport) environment.

Computer-Assisted Passenger Screening and Positive Passenger-Bag Matching

CAPS appears to be an effective way to screen passengers to identify selectees who require further security measures, such as bag matching or bag screening. The committee anticipates that PPBM combined with CAPS will be an effective tool for improving aviation security. Despite the positive attributes of CAPS, the committee is concerned that the FAA has not demonstrated a measure for characterizing quantitatively the effectiveness of CAPS. In addition, there are some methods by which a CAPS selectee could bypass PPBM.

Recommendation. Computer-assisted passenger screening (CAPS) should continue to be used as a means of identifying selectee passengers whose bags will be subject to positive passenger-bag matching (PPBM) or screening by explosives-detection equipment, or both. PPBM combined with CAPS should be part of the five-year plan recommended below. Passengers that have been designated as selectees at the origination of their flights should remain selectees on all connecting legs of their flights.

2. Determine how the technologies referred to in paragraph (1) could be used more effectively to promote and improve security at airport and aviation facilities and other secured areas.

Progress in the Deployment of Aviation Security Equipment

The committee concluded that the FAA/SEIPT, the airlines, airports, and associated contractors have gained significant experience from the initial deployment of security equipment and procedures, and the current implementation of security equipment does not appear to have interfered unreasonably with airline operations. Most importantly, in the collective opinion of the committee, the deployment of security equipment has improved aviation security (see Figure 2). The committee believes that continued emphasis on, funding of, and deployment of security equipment will further enhance aviation security. Future deployments should be more efficient if they are based on the experience from the initial deployment.

Recommendation. The U.S. Congress should continue to fund and mandate the deployment of commercially available explosives-detection equipment through the FAA/SEIPT. Continued deployments will increase the coverage of domestic airports and eventually provide state-of-the-art security equipment system wide. Further deployments can improve aviation security in the short term and provide the infrastructure for mitigating potential threats in the long term.

Figure 2 - Comparative contributions (notional) to the SEF of various security technologies and training (EDS = explosives detection system, TEDD = trace explosives detection device, HULD = hardened unit loading device).

Note: The baseline illustrated in Figure 2 represents the state of aviation security prior to the 1997 deployment of EDS. Each additional point (e.g., EDS) represents the notional improvement in aviation security afforded by the deployment (or future deployment) of technology, combinations of technology, and implementation of training. The security enhancement factor (SEF) is a measure of the improvement of aviation security and is discussed in detail in the first report.

Operator Performance

Human operators are integral to the performance of all deployed explosives-detection equipment. Because fully automated explosives-detection equipment will not be developed in the foreseeable future, particularly with respect to alarm resolution, human operators will continue to be immensely important to realizing the full potential of deployed security hardware. The TAAS analysis presented in this report quantifies the impact of the operator on the SEF. Certification testing of explosives-detection equipment, however, does not include testing of human operators. Current testing only defines the operational capability (or performance) of the equipment.

Recommendation. The FAA should institute a program to qualify security-equipment operators to ensure that the human operator/explosives-detection system (EDS)combination meets the performance requirements of a certified EDS. This program should include the definition of operator performance standards and a means of monitoring operator performance.

Measuring Operational Performance

Because of the paucity of operational data for deployed explosives-detection equipment, the committee found it impracticable to characterize the deployment status of security equipment and processes quantitatively. The data are insufficient both for the equipment and for operator performance, and no quantitative measures of the effectiveness of the total security system (e.g., TAAS) were provided to the committee. The majority of data focused on subsystems, such as bulk explosives-detection systems. A thorough assessment of equipment and system performance requires well defined performance metrics and the collection of data. The committee concluded that the FAA has not defined adequate performance metrics for security subsystems (e.g., TEDDs) or for the TAAS.

Recommendation. The FAA should make a concerted effort to define operational performance metrics for security subsystems and for the total architecture for aviation security (TAAS). The FAA should also create an action team to systematically collect operational data, which should be used to optimize the TAAS, as well as to identify and correct substandard performance of equipment and operators. The data collected would also provide insights into the deployment and use of equipment in the future.

Measuring Security Enhancement

Besides the dearth of operational data and total-system performance metrics, the FAA has not defined an overall measure of security enhancement. The primary performance measure for the TAAS is, of course, protection against the threat of explosives. Consequently, the committee believes the critical factor in assessing the performance of the TAAS is the measure of false negatives (i.e., unidentified bags that contain explosives). The committee defined improved performance (i.e., the SEF) as the ratioof the number of simulated bombs that defeat the baseline security system to the number of simulated bombs that defeat the newly deployed system.

Recommendation. The FAA should formulate a security enhancement factor (SEF) for the integrated total architecture for aviation security systems. The SEF should be calculated from data collected during operational testing. No classified SEF measures should be published and used as a project-control and management-control tool. The SEF would provide the FAA with a quantitative measure of the impact of security equipment and procedures.

Five-Year Deployment Plan

Decisions based on systems of systems analysis (e.g., TAAS) involve both management and cost factors, which are airport and airline specific. Stakeholder7involvement, therefore, will be crucial for the development of an effective deployment strategy. Furthermore, airline and airport buy-in will be critical to the successful implementation of the deployment strategy. The FAA did not provide the committee with a long-range (five-year) TAAS deployment plan developed jointly and agreed to by the FAA and other stakeholders. Thus, the committee concluded that the FAA has not obtained comprehensive airline buy-in for a long-term deployment plan that addresses all of the relevant issues, such as operator training, the optimal location of detection equipment, and the operational deployment of HULDs.

Recommendation. In cooperation with the other stakeholders, the FAA should develop a five-year joint-deployment plan that includes cost, stakeholder responsibilities, quality measures, and other important factors. This plan should be a living document that is formally updated annually. Buy-in from all stakeholders will be necessary for the plan to be effective.

3. Assess the cost and advisability of requiring hardened cargo containers to enhance aviation security and reduce the required sensitivity of bomb-detection equipment.

Two Galaxy HULDs (both LD-3 size) that conform to NAS-3610-2K2Cairworthiness criterion have passed the FAA blast and shockholing8 tests. The initial phase of operational testing of the HULDs have revealed shortcomings that have resulted in design changes. The redesigned HULDs have passed FAA blast tests and are now undergoing operational testing with two airlines. It should be noted that, as currently configured, the HULD can only be used on wide-body aircraft. Thus, no HULD concept for narrow-body aircraft has passed the FAA test, although 75percent of the aircraft in service (as of 1994) are narrow-body aircraft, and more than70 percent of bombing attempts have been against narrow-body aircraft.

The committee's greatest concern is that research on HULDs has not been conducted on a system-of-systems (SOS) basis and has not involved all of the stakeholders, mainly the airlines. So far, HULDs have largely been developed and designed as single, stand-alone entities. Limited research has been done on their role as part of a TAAS. Coordination with the airlines, airports, and aircraft manufacturers has been focused mainly on specific designs and utility requirements rather than on the interactions, boundary conditions, and trade-offs (including cost and operational considerations) of using HULDs along with other security measures, such as passenger profiling and baggage screening. The committee believes that alternative HULD designs may be more practical than existing designs in the TAAS context.

Recommendation. The FAA should continue to support research and development on hardened unit loading devices (HULDs), including ongoing operational testing. If the FAA recommends, mandates, or regulates the use of HULDs, explosion-containment strategies for narrow-body aircraft, including the development of narrow-body HULDs and cargo-hold hardening concepts, should be investigated. However, the FAA should not deploy HULDs unless they are part of the TAAS joint five-year deployment plan.

4. On the basis of the assessments and determinations made under paragraphs (1), (2), and (3), identify the most promising technologies for improving the efficiency and cost-effectiveness of weapons and explosive detection.

The data were not sufficient for a comprehensive assessment of available technologies for improving aviation security. Therefore, at this time, the committee is not able to identify or recommend the most promising technologies for improving the efficiency and cost effectiveness of weapons and explosives detection. If the recommendations in this report are followed, these data will become available for subsequent assessments.

Mr. Chairman, that concludes my prepared testimony. I would be happy to address any questions at this time.

1. NRC (National Research Council). 1999. Assessment of Technologies Deployed to Improve Aviation Security: First Report. National Materials Advisory Board. Washington, D.C.: National Academy Press. Available on line at: http://books.nap.edu/catalog/9726.html.

2. The following terminology is used throughout this testimony. An explosives-detection system is a self-contained unit composed of one or more integrated devices that has passed the FAA's certification test. An explosives-detection device is an instrument that incorporates a single detection method to detect one or more explosive material categories. Explosives-detection equipment is any equipment, certified or otherwise, that can be used to detect explosives.

3. In this testimony, bulk explosives include all forms and configurations of an explosive at threat level (e.g., shaped explosives, sheet explosives, etc.).

4. DOT (U.S. Department of Transportation). 1998. Aviation Security-Federal Aviation Administration. Washington, D.C.: U.S. Department of Transportation, Office of the Inspector General,. Available on line: http://www.dot.gov/oig/audits/av1998134.htm

5. GAO (General Accounting Office). 1998. Aviation Security: Implementation of Recommendations Is Under Way, But Completion Will Take Several Years.GAO/RCED-98-102. Washington, D.C.: General Accounting Office. Available on line at: http://www.gao.gov/AIndexFY98/abstracts/rc98102.htm

6. White House Commission on Aviation Safety and Security. 1997. Final Report to the President. February 12, 1997. Available on line at: http://www.aviationcommission.dot.gov/

7. In this report the term stakeholder includes the FAA, the airlines, and the airports. Although there are certainly other stakeholders in aviation security, these three will have the most influence on the deployment strategy for aviation security equipment.

8. A shockholing (or fragmentation) test measures the ability of a HULD to prevent perforation of its walls by a metal fragment traveling at a relatively high velocity.

RSS News Feed | Subscribe to e-newsletters | Feedback | Back to Top