In early 2020, Carol Tenopir (University of Tennessee) and Dave Nicholas (CIBER) completed a four-year longitudinal study of early career researcher (ECR) practices across the natural and social sciences.  Drawing on more than 350 hours of interviews with 100 early career researchers in China, France, Poland, Malaysia, Spain, Russia, the U.K. and the U.S., the findings ranged from the expected to the surprising: for instance, young researchers were nearly universally indifferent to using altmetrics to measure scholarly impact, and they showed little interest in publishing in Open Access journals, despite widespread dissatisfaction with existing academic regimes of publication and promotion.  Funds from this grant will support the extension of Tenopir and Nicholas’s work, with an emphasis on how the COVID-19 pandemic is affecting the research and communications practices of young researchers around the globe.

This grant funds a project by computer scientist Jeannette Wing, Director of the Columbia University Data Science Institute and Professor of Computer Science, to adapt "formal methods” (the representation of computer science systems as mathematical objects) to AI systems.  Once the AI system, the input data, and the desired trust property are formally specified, the AI system can then be analyzed using mathematics, allowing a skilled analyst to rigorously prove or disprove statements about the system being represented. The technique holds obvious appeal for those concerned about the trustworthiness of AI systems, since a formal methods analysis has the potential to reveal how an AI system would or would not behave in novel situations.  Grant funds will support Wing’s attempts to extend formal methods theory to AI systems, including how to formally specify properties of AI systems like fairness, privacy, and robustness.  A particular focus of Wing’s work will be to better formally understand the relationships among such properties, in order to identify and generalize their commonalities and differences.  Wing will also work on trying to use formal methods to characterize, with respect to these trust properties, the relationship between AI systems and the datasets used for training and testing them.

Open and inexpensive hardware has the potential to revolutionize the creation and deployment of sensors and other scientific instruments, expanding access and lowering barriers to innovation in data-driven research methods.  Much of the activity within the open hardware movement has been on expanding the distributed production of hardware, through tools like the open licensing of hardware design and the creation of open 3-D      printing templates for instrument parts.  There has been comparatively less emphasis, however, on how to measure and ensure quality control in a distributed production process.  The widespread availability of inexpensive sensors will only revolutionize science, after all, if the sensors actually work.  This project by University of Washington researcher Nadya Peek will  improve our understanding of quality control in distributed manufacturing processes.  Over the course of the grant, Peek will engage in four streams of activity aimed at filling gaps in current open hardware calibration practices. First, she will develop a generalizable format for documenting the theoretical capabilities of a production machine like a consumer-grade 3D printer.  Second, once this format is created, Peek will use it to develop calibration software capable of verifying that a specific instance of that machine is performing to expectations and within acceptable error parameters.  Third, Peek will develop new software to monitor such machines in real time, ensuring that they are maintaining precision and calibration through the production process.  Fourth, she will develop low-barrier procedures for testing the precision and quality of the final output. In addition, Peek will also field a survey questioning how researchers in the open hardware community are adapting their distributed production processes in response to the shutdowns caused by the COVID-19 pandemic.  

The Internet Age is quickly giving way to the Age of the Algorithm.  Decision-makers of all kinds are increasingly turning to complex algorithmic methods to help them allocate resources, set policies, and assign risk.  Banks use algorithms to figure out how likely someone is to default on a loan. Online retailers use algorithms to decide which ads to display on your phone.  Pollsters use algorithms to determine who is and who is not likely to vote. Increasing reliance on algorithmic verdicts comes with risks of its own, however.  The worry is not so much that the algorithms might get things wrong—human judgement, after all, is hardly error free--but they might get things systematically wrong, disfavoring one group of people over another for arbitrary or irrelevant reasons.  The worry is that we might build algorithms, in other words, that are unfair. This grant funds efforts by a team led by Harvard computer scientist Cynthia Dwork that aim to address this issue. Dwork’s plans involve constructing new theoretical frameworks—based on rigorous mathematical notions called pseduorandomenss, latitude, and multicalibration--that can be used to define and evaluate whether an algorithm is fair or not.  Grant funds will allow Dwork to fully develop her theory, build some algorithms that meet that those characteristics described, and test them to see if they indeed perform as theory predicts.  If successful, the effort would constitute a significant stride forward in our understanding of an increasingly essential cog in the machinery of modern life.