This grant provides two years of operational support for the Center on Regulation and Markets, a project of the Brookings Institution that provides independent, non-partisan research on regulatory policy, applied broadly across microeconomic fields. Led by Sanjay Patnaik, the Center creates and promotes independent economic scholarship to inform regulatory policymaking, the regulatory process, and the efficient and equitable functioning of markets. Research supported by the Center addresses a number of pressing issues in regulatory economics, including financial markets, emerging technologies, consumer protection, regulatory processes, data privacy, common ownership, and how to accurately measure market power. Grant funds will allow the Center to commission four policy papers and four policy briefs on topics in regulatory economics; hold eight events aimed at disseminating research to academics, policymakers, regulators, the press and the public; and maintain and update the Center’s website as a general dissemination hub for information about the Center, its research, events, and other activities.

This grant funds work by a team led Francesca Dominici and Jose Zubizarreta to develop new methods and techniques that will increase the robustness and power of randomized controlled trials (RCTs) as a method for investigating causal relations across a diverse range of phenomena. Long-regarded as the gold standard in social scientific research, the randomized controlled trial has virtues in abundance. By randomly sorting participants into control and treatment groups, researchers using RCTs can, in theory, ensure that these groups are statistically indistinguishable. This allows them to conclude that differences later observed between these two groups must have been caused by the treatment. This works beautifully in principle. In practice, however, drawing causal inferences using RCTs can be bedeviled by a number of factors, all involving how statistical averages never tell the whole story. When the population under study is very diverse, for instance, randomly sorting participants into control and treatment groups may be insufficient to ensure the two groups are identical across all variables. In other instances, control and treatment groups may be insufficiently isolated from one another, allowing outcomes caused by the treatment to spillover into the control group. In other cases, the effect of a treatment within the treatment group may be unequally distributed. A treatment that benefits a few people greatly while leaving most people worse off, say, might appear to have a positive benefit on average, leading researchers to miss important facts about how that average benefit is generated. Dominici and her team will develop and test new statistical methods that, if successful, will help researchers design RCTs in ways that head off each of these problems and allow the design of RCTs that can be more reliably used to make causal inferences. Their results will be distributed through academic papers, talks at professional meetings, and through open-source software tools available to be downloaded by researchers.

The mathematical theory of differential privacy describes methods and practices that can be implemented that allow researchers to query datasets with sensitive information while monitoring how much each query threatens the privacy of the individuals contained in the dataset. Differentially private methods are the current cutting edge of privacy-protecting science, yet, they are often mathematically complex and difficult to implement for those not versed in them. Widespread use of these methods will require mediating institutions that lower the cost of adoption, trusted places where researchers can download easy-to-install and easy-to-use software applications that will allow them to apply differentially private firewalls to sensitive data. In response to this need, Harvard computer scientist Salil Vadhan has created OpenDP, a dedicated community of theorists, engineers, practitioners, and privacy experts that are aiming to increase adoption of differential privacy by producing an open source suite of flexible, tested, and industrial-strength software components that makes implementing differential privacy both straightforward and trustworthy. Funds from this grant will support the effort, allowing Vadhan to further develop the library of general-purpose differential privacy algorithms, attract new experts to the collaboration, form new partnerships with corporations interested in protecting sensitive data, promoting awareness of the collaboration and its tools, and holding an annual meeting of stakeholders and users from academia, government and industry.

Experiments conducted online, using subjects that participate via web interface instead of physically traveling to a lab, have significant advantages over their in-person counterparts. They are cheaper to field, for instance, and they can draw from a more diverse pool of potential test subjects, making inferences from their findings more robust. Setting up such virtual experiments, however, is not easy. Existing software support packages for online experiments have been developed as “end-to-end” platforms that are not very flexible, much less interoperable. Customizing this software to meet the eccentricities of any given experiment often requires special programming skill and patience. Funds from this grant support a project, led by Duncan Watts at the University of Pennsylvania and Abdullah Almaatouq at MIT, that seeks to make fielding online experiments easier for researchers of all kinds. Watts and Almaatouq aim to develop the first modular virtual experiment platform, one that subdivides an experimental design into independent, though interoperable, parts with standard interfaces. Friendly graphical controls will enable researchers and administrators to customize, reuse, and improve each module of an experiment without writing new code. One feature, for example, will be automatic recruiting tools that simplify the location and retention of participant panels that are large, diverse, and representative. Grant funds will allow Watts and Almaatouq to develop and launch an entire virtual environment that will facilitate running social science experiments that are faster, cheaper, more scalable, more complex, and more realistic than could take place in a physical laboratory. All code for this software environment, in addition to accompanying documentation, tutorials, and webinars, will be made freely available through a professional archive that makes it easy for experimenters to both preregister their experiments as well as to deposit their code, data, and documentation. Additional grant funds will support outreach and adoption activities designed to encourage use of the platform and to begin to build an open-source community of developers devoted to its maintenance and improvement.

This grant provides funding for the continuation and expansion of a doctoral fellowship program focused on training an interdisciplinary cohort of early career researchers to conduct scholarship in energy data analytics. Implemented by Duke University’s Energy Data Analytics Lab, the program provides participants from multiple fields with opportunities to work on extended research projects involving the collection and analysis of data related to the energy system. In this expanded program model, fellows will be drawn from six participating universities from the region, including Duke, University of North Carolina (UNC)-Chapel Hill, UNC-Charlotte, UNC-Greensboro, North Carolina State University, and North Carolina A&T State University, a Historically Black College and University (HBCU). Fellows are drawn from a diverse variety of institutional backgrounds, including computer science; environmental policy; earth and ocean sciences; and electrical, civil, computer, and environmental engineering.  Fellows work with a team of advisors to develop original data-driven research projects that they work on during the summer months.  Past participant projects include using satellite imaging to asses energy loss at solar energy facilities, constructing neural networks for use in predicting energy demand, modeling the energy consumed in the production of consumer products, and enhancing image datasets to allow for the identification of buildings with poor insulation. Six fellows will be supported in each year of the three-year grant period. Grant funds will primarily support student stipends, with additional funds devoted to various support activities, including mentorship, periodic workshops for student participants, and a conference allowing students and other researchers to showcase their work and allowing for networking opportunities with practitioners

This grant provides support to organize an annual, intensive, week-long training program at the University of California Berkeley introduces second- and third-year doctoral students interested in energy and environmental economics to state-of-the-art research in the field.  Each year approximately 60 doctoral students from universities across the country arrive in Berkeley to receive rigorous training in energy and environmental economics and have the opportunity to interact with leading faculty in the field. Nearly a semester’s worth of content is presented in a condensed period of time, helping students participants gain the knowledge necessary to undertake more in-depth research when they return to their home institutions. Grant funds will provide support for a number of student scholarships, with a specific focus on increasing the number of women and underrepresented minorities participants and strengthening mentorship activities over the course of the training program.

Individual states play a significant role in setting and implementing energy policies of all kinds, yet no comprehensive database exists of state-level energy policy initiatives and projects.  Without such a database, it is very difficult to understand how state policies harmonize, augment, or conflict with one another (or with federal policy), how state policies could be altered to better achieve overall energy goals, or how state policies differ (or do not) by region, geography, culture, industrial composition, and much more besides. This grants funds an initiative to address this information gap by creating an inventory that makes systematic information about state energy policies available to the research and policy communities. It will also engage scholars and decision-makers to use this new resource and begin to rigorously analyze the connections, overlaps, and relationships between different state energy policy regimes. After constructing the inventory, the team will analyze state policies across across four dimensions: emissions reduction targets, economic outcome targets, resilience planning, and cross-governmental coordination. Research findings will be published as reports and articles and disseminated at a final workshop and shared broadly, with a focus on engaging state and federal decision-makers.

The Journal of Open Source Software (JOSS) is a peer-reviewed academic journal that specializes in the publication of articles about open source scientific research software.  In JOSS, authors can submit a software project along with a description of its scientific use, capacities, limitations, the technical resources required to deploy it, how to access it, and its associated technical documentation.  Following submission, a ’software-native’ review (on GitHub) workflow enables community members to perform code review as well as review documentation and associated metadata. Once published, scientists who use, reuse, adapt, or modify a piece of JOSS-published software in their own research can then cite the JOSS Document Object Identifier (DOI), giving recognition to the software’s developers and creating corresponding professional incentives for scientists to contribute to the development of open source research software.  Funds from this grant support efforts to allow JOSS to better serve its authors and readers through improving and documenting a number of elements of the JOSS technical platform as well as generalizing the software review components for peer review of software outside of JOSS. Funded activities include planned improvements to the “bot” that automates much of the coordination and technical checking of software submitted to the journal, as well as the creation of developer guides, deployment recipes, and a reviewer management system.  The underlying JOSS infrastructure is itself an open source project, allowing other organizations interested in conducting automated software review to benefit from the JOSS team’s work.  

By combining archived software code with information on the operating system, application, drivers and other information about the computational environment in which a software program was originally run, The Emulation as a Service Infrastructure (EaaSI) can trick software into thinking it’s being run on the hardware for which it was built.  The result is a sort of software time machine, allowing historians and researchers to interact with decades old software just as users at the time interacted with it.  Even better, EaaSI’s emulations require no special equipment to execute.  Anyone with a web browser can connect to and use the service.  Funds from this grant, led and administered by Yale University Library (along with partner funding from the Andrew W. Mellon Foundation), support the continued expansion and development of the EaaSI ecosystem.  Planned activities include the introduction of new features, like the ability to model networked resources within emulated software and the emulation of mobile phone and tablet apps, as well efforts to grow the number of institutions hosting EaaSI nodes and to provide enhanced training and documentation for users.

Open source software (OSS) is an increasingly vital component of the scientific research enterprise, used in one form or another at every point in the research pipeline, from instrument calibration, to data collection and cleaning, to analysis and visualization, to archiving.  The centrality and importance of OSS has led to the realization within academic institutions of the need for formal mechanisms to identify and support those OSS projects most central to its researchers.  One model being explored is the creation of university Open Source Programs Offices (OSPO), special intra-university bodies charged with the support of important open source software.    This grant provides funding for the Open Source Contributor Fund at the Johns Hopkins University, a pilot initiative designed to enhance and deepen the university’s support for and engagement with faculty working on open source software projects.  Spearheaded by Associate Dean for Research Management G. Sayeed Choudhury out of the university’s new Open Source Programs Office, the Fund will make small grants of $10,000 to those open source software projects deemed to be most important to campus researchers.  The 16 projects supported over the grant period will be selected through a combination of voting and data analysis of research software dependencies. In addition to surfacing appropriate projects for support, the nomination and voting process will be used to canvass the use, development, and maintenance of open source software across Johns Hopkins. Choudhury and his team will also produce a playbook and other open source tools for use by other institutions who wish to implement similar programming in support of open source development.