Artificial intelligence (AI) algorithms are being built and trained to perform a wide variety of tasksСrecognizing faces, identifying objects in photos, processing natural language by extracting concepts from text. Once a system is built and trained, however, how do we know how well it performs relative to other such systems? How do we know if the data used to train the system reflect the context in which the system will be used? To answer these questions, we need to scrutinize the training datasets that are used to construct AI systems, and the benchmarking datasets against which these systems are assessed. This grant supports work by Meredith Whittaker and Kate CrawfordСthe co-founders of the AI Now Institute at New York UniversityСand NYU Law professor Jason Schultz. Over the course of three years, Whittaker, Crawford, Schultz, and their team will dig deeply into the history, design, and technical details of some of the most foundational AI datasets, investigating where they came from, how they have evolved, and how they have been used over time. They will use these findings to catalyze a broader conversation about how to understand and appropriately govern the AI systems that are informed by these datasets. The grant outputs will include multiple papers produced for both academic and lay audiences, visualizations of the provenance and uses of specific datasets, and workshops that will bring together the growing community of researchers studying the data that underpins AI research.

This grant funds a project by Anne Bowser, Director of Innovation at the Woodrow Wilson International Center for Scholars, to conduct a comprehensive review of the use of low-cost, including open source, hardware in scientific research. Open hardware refers to the licensing of the design specifications of a physical object in such a way that the described object can be created, modified, used, or distributed by anyone. Open hardware sensors or other instruments present an attractive opportunity to expand the frontiers of scientific research by dramatically lowering the costs of instrumentation. Despite this promise there is, as yet, no comprehensive account of the full range of low-cost and open source hardware solutions; how hardware is being used by researchers and public policy communities; what, if any problems have arisen for those using open hardware related to data quality, governance, and standards; and what institutions and norms are needed to encourage adoption. Bowser and her team will conduct a wide-ranging review of low-cost hardware and the open hardware movement, combining broad landscape synthesis and convenings with commissioned reports on critical issues like data quality, governance, and the relationship between open hardware and other open paradigms.

Whether you call it machine learning, deep learning, or AI, a new set of methods at the interface of statistics and computer science are being applied to research across the sciences. A consequence of the excitement about these new methods is that disciplinary researchers eager to use them in their research must both get up to speed quickly and maintain an awareness of a new literature, one which is moving at high volume and velocity. Increased interest in the AI literature, however, comes just as that literature is getting harder to read thanks to a combination of short publish-response cycles and rapidly evolving norms about what should be cited and explained in a given paper. This grant funds a project by computer scientist Marti Hearst to develop interfaces to the AI literature that offer additional context and support for readers not deeply acquainted with the field. Hearst’s lab will develop algorithms and software to help readers see the meanings of symbols and terms anywhere in the text of a given article, regardless of where they are defined, and pull in explanations from papers in the co-citation network of the paper being read where definitions are not present in the text itself. The resulting software, implemented in a lightweight interface that integrates with PDF readers to ensure wide adoption, will be of value to researchers across the sciences who are adopting machine learning methods.