Sec. 102. Program on artificial intelligence testbeds
928 words·~4 min read·
/bill/118/s/4178/is/section-102A research copy — for the controlling text, always check the official state or federal source. Not legal advice.
In this section: The term appropriate committees of Congress means— the Committee on Commerce, Science, and Transportation and the Committee on Energy and Natural Resources of the Senate; and the Committee on Science, Space, and Technology of the House of Representatives. The term Director means the Director of the National Science Foundation. The term Institute means the Artificial Intelligence Safety Institute established by section 101. The term Secretary means the Secretary of Energy.
The term Under Secretary means the Under Secretary of Commerce for Standards and Technology. Not later than 180 days after the date of the enactment of this Act, the Under Secretary shall, in coordination with the Secretary and the Director, establish and commence carrying out a testbed program to encourage collaboration and support partnerships between the National Laboratories, the National Institute of Standards and Technology, the National Artificial Intelligence Research Resource pilot program established by the Director of the National Science Foundation, or any successor program, and public and private sector entities, including companies of all sizes, to conduct research and development, tests, evaluations, and risk assessments of artificial intelligence systems, including measurement methodologies developed by the Institute.
In carrying out this program, the Under Secretary shall, in coordination with the Secretary— use the advanced computing resources, testbeds, and expertise of the National Laboratories, the Institute, the National Science Foundation, and private sector entities to run tests and evaluations on the capabilities and limitations of artificial intelligence systems; use existing solutions to the maximum extent practicable; develop automated and reproducible tests, evaluations, and risk assessments for artificial intelligence systems to the extent that is practicable; assess the computational resources necessary to run tests, evaluations, and risk assessments of artificial intelligence systems; research methods to effectively minimize the computational resources needed to run tests, evaluations, and risk assessments of artificial intelligence systems; consider developing tests, evaluations, and risk assessments for artificial intelligence systems that are designed for high-, medium-, and low-computational intensity; and prioritize identifying and evaluating scenarios in which the artificial intelligence systems tested or evaluated by a testbed could be deployed in a way that poses security risks, and either establishing classified testbeds, or utilizing existing classified testbeds, at the National Laboratories if necessary, including with respect to— autonomous offensive cyber capabilities; cybersecurity vulnerabilities in the artificial intelligence software ecosystem and beyond; chemical, biological, radiological, nuclear, critical infrastructure, and energy-security threats or hazards; and such other capabilities as the Under Secretary determines necessary.
In carrying out the activities required by subsection (c), the Under Secretary shall, in coordination with the Secretary, take under consideration the applicability of any tests, evaluations, and risk assessments to artificial intelligence systems trained using primarily biological sequence data, including those systems used for gene synthesis. The Under Secretary, in collaboration with the Secretary, shall develop metrics— to assess the effectiveness of the program in encouraging collaboration and supporting partnerships as described in subsection (b); and to assess the impact of the program on public and private sector integration and use of artificial intelligence systems.
In carrying out the program required by subsection (a), the Under Secretary may, in collaboration with the Secretary and the Director, use a program that was in effect on the day before the date of the enactment of this Act. Not later than 3 years after the start of this program, the Under Secretary shall, in collaboration with the Secretary— evaluate the success of the program in encouraging collaboration and supporting partnerships as described in subsection (b), using the metrics developed pursuant to subsection (e); evaluate the success of the program in encouraging public and private sector integration and use of artificial intelligence systems by using the metrics developed pursuant to subsection (e); and submit to the appropriate committees of Congress the evaluation supported pursuant to paragraph
(1)and the findings of the Under Secretary, the Secretary, and the Director with respect to the testbed program. In carrying out subsection (b), the Under Secretary shall consult, as the Under Secretary considers appropriate, with the following: Industry, including private artificial intelligence laboratories, companies of all sizes, and representatives from the United States financial sector. Academia and institutions of higher education. Civil society. Third-party evaluators. In carrying out the program under subsection (b), the Under Secretary shall, acting through the Director of the Institute and in coordination with the Secretary of Energy, carry out a test program to provide vendors of foundation models the opportunity to voluntarily test foundation models across a range of modalities, such as models that ingest and output text, images, audio, video, software code, and mixed modalities, relative to the Artificial Intelligence Risk Management Framework, by— conducting research and regular testing to improve and benchmark the accuracy, efficacy, and bias of foundation models; conducting research to identify key capabilities, limitations, and unexpected behaviors of foundation models; identifying and evaluating scenarios in which these models could pose risks; establishing reference use cases for foundation models and performance criteria for assessing each use case, including accuracy, efficacy, and bias metrics; enabling developers and deployers of foundation models to evaluate such systems for risks, incidents, and vulnerabilities if deployed in such use cases; coordinating public evaluations, which may include prizes and challenges, to evaluate foundation models; and as the Under Secretary and the Secretary consider appropriate, producing public-facing reports of the findings from such testing for a general audience. Nothing in this section shall be construed to require a person to disclose any information, including information— relating to a trade secret or other protected intellectual property right; that is confidential business information; or that is privileged.