Sec. 201. Support for evaluation of artificial intelligence automation
578 words·~3 min read·
/bill/119/s/3339/is/section-201A research copy — for the controlling text, always check the official state or federal source. Not legal advice.
Not later than 270 days after the date of the enactment of this Act, the Secretary of Commerce, acting through the Director of the National Institute of Standards and Technology (in this section referred to as the Director ), shall commence carrying out at least 1 prize competition under section 24 of the Stevenson-Wydler Technology Innovation Act of 1980 ( 15 U.S.C. 3719 ) to develop benchmarks or similar reproducible methods to quantitatively measure the ability of artificial intelligence to automate or augment tasks or occupations, with the primary purpose of improving forecasts of the impacts that artificial intelligence may have on workers and the retraining needs of workers.
In carrying out the prize competition required by subsection (a), the Director shall consult with the Secretary of Commerce, the Secretary of Labor, the Commissioner of the Bureau of Labor Statistics, and the Director of the National Science Foundation. The Director shall develop the structure and evaluation criteria for a prize competition carried out under subsection
(a)and a companion grant or cooperative agreement awarded under subsection
(d)in accordance with the primary purpose described in subsection (a). The Director may structure a competition under subsection
(a)into 1 or more categories, including by the occupation impacted or the capability domain of the artificial intelligence. In selecting the categories under paragraph (2), the Director shall consider— the informativeness of a category in evaluating the direct or indirect impacts on labor markets, which may include factors such as the number of individuals directly or indirectly impacted by the automation or augmentation and the economic security or ability to find new employment opportunities of the individuals impacted; the quality or efficacy of the submissions for a category; the potential for a category to result in the development of benchmarks that complement existing benchmarks; the potential for a category to result in the development of benchmarks that are underfunded by private contributions relative to the public value of the benchmarks; and other factors determined appropriate by the Director, taking into account input gathered under section 101. The Director may structure a competition under subsection
(a)into 1 or more phases (such as design and prototype stages). In developing the evaluation criteria for the benchmarks or similar methods submitted for prize competitions under subsection (a), the Director shall, to the extent practicable, seek to avoid or mitigate common problems affecting the value and feasibility of artificial intelligence benchmarks, including— training data contamination that is difficult to detect; rapid loss of discriminatory value over time due to a metric having a low ceiling for performance; and other common problems, including problems identified in the input collected under section 101. In carrying out this section, the Director may, in addition to carrying out a prize competition under subsection (a), award a grant or enter into a cooperative agreement to support the design, construction, validation, and maintenance of a benchmark or similar reproducible method described in subsection (a), including support for— data collection and labeling; evaluator training and rubric development; and third-party replication and inter rater reliability testing. The Director may enter into contracts, cooperative agreements, or other agreements with for-profit or nonprofit entities, or State, territorial, local, or Tribal agencies, to design and administer prize competitions under subsection
(a)and related grant and cooperative agreement activities under subsection (d). There is authorized to be appropriated to the Secretary of Commerce to carry out this section $7,000,000 for the period of fiscal years 2026 through 2030.
Connectionstraces to 1
Traces to 1 document
U.S. Code
Citation graph
cites case law
Sec. 201
Support for evaluation of artificial intelligence automation
Cites 1Cited by 0 across 0 sources