This grant supports the Digital Public Library of America to expand its nationwide service hub network. Service hubs are on-ramps in each state for uploading and sharing digital content from the smallest private collection in a remote rural library to the largest state library or museum. As such, they are the key to DPLA's grass-roots, bottom-up, decentralized approach to building a national digital library. Hubs host locally provided digital content for the DPLA, correct and add metadata to uploaded items, coordinate local events and public outreach, and collaborate with state cultural institutions on digital initiatives. Grant funds will allow the DPLA to add eight new service hubs to its current roster of 15, increasing coverage by 50 percent and moving the institution closer to its goal of being a truly national digital library. Funds from this grant also support a DPLA initiative to partner with authors, publishers, libraries, and the White House to launch a new service network that provides free eBooks to children.

What do behavioral economists really know? Lessons learned so far seem more about isolated, but intriguing, examples rather than coherent or unifying principles. What counts as accepted doctrine is based almost exclusively on empirical results about particular phenomena such as loss aversion, probability weighting, altruism, hyperbolic discounting, and social comparisons. One would expect, therefore, that experts would be rather good at predicting the outcomes of standard experiments about standard topics in behavioral economics. This grant funds a research project by Devin Pope of Chicago and Stefano DellaVigna of Berkeley that test that hypothesis. First, Pope and DellaVigna will ask experts to forecast the effects of 17 different behavioral interventions or “nudges” in standard, simple, familiar, and carefully specified experiments. Second, they will run these experiments as described in a common setting. A large number of subjects will be asked to perform an effortful 10-minute task online. Each will be assigned to one of the 17 different framings, incentive structures, or other treatments. Just by keeping everything else equal except these behavioral interventions, the experimenters will be able to draw conclusions about the relative magnitudes and probabilities of various effects. Third, they will compare the expert forecasts with the experimental results. It is possible, of course, that all the predictions will turn out to be quite accurate—or not. In any case, such an exercise should help identify what behavioral economists do agree upon and, therefore, what we have learned from behavioral economics.

Among big questions about the economics of science, two of the most important and challenging concern investments in research and development (R&D): How do the inputs to R&D map into scientific breakthroughs? And how do the inputs to R&D map into broader social returns? This grant funds efforts by Benjamin Jones of Northwestern University to make fresh progress on each of these questions. First Jones will focus on the productivity of scientific teams, investigating how the characteristics of individual team members contribute to overall performance in different contexts. We know little about what makes effective scientific collaboration. For theoretical work, perhaps the strength of the strongest researcher drives results; in the lab, perhaps the strength of the weakest researcher matters most; and, in other situations, it may be some kind of average over everyone. Jones will use output and productivity data on scientific team composition to try to understand how these different skills and training fit together to influence scientific productivity. In a second effort, Jones will investigate the time delays between investments in and payoffs from R&D. Starting with NSF and NIH grant numbers, he will link newly available microeconomic data that trace how long it takes in various fields for grants to turn into papers, for papers to turn into patents, and for patents to turn into adopted technologies. Jones will then use these data to calculate societal returns to government investment in science.

This grant supports University of Pennsylvania computer scientist Aaron Roth in his work to develop, analyze, and evaluate “differentially private” algorithms for use in scientific discovery. First developed by mathematicians concerned about privacy, differentially private algorithms are ways of querying sensitive datasets. An algorithm or database query is “differentially private” if the results it returns would be provably the same even if an individual record were randomly replaced by another record in the queried dataset. Since the results such algorithms return do not depend on whether a given record is or is not included in the dataset, one cannot reverse engineer who is in the dataset from the results it generates. The privacy of the data is thereby protected. As it happens, this privacy protecting feature has uses outside the concern to protect privacy. Differentially private algorithms also prevent data mining and overfitting. Since differentially private algorithms produce the same results regardless of whether a given observation is randomly replaced by another, it is difficult to use them to craft results tailored to the particularities of the data you happen to have collected. At present, however, differentially private algorithms are more exciting in theory than in practice. They tend to be laborious and slow. What’s needed is further development and testing of such algorithms with scientific applications in mind. Dr. Roth is working on just such an approach, trying to develop practical applications of differentially private algorithms that are streamlined and reliable enough to be used in everyday scientific practice and analysis.

This grant funds a project lead by California Institute of Technology economist Colin Camerer to attempt to replicate the findings of 18 seminal papers in economics. Working with the original authors, Camerer has selected highly influential, highly cited papers that all deal with between-subject treatment effects that appeared between 2011 and 2014 in either the American Economic Review or the Quarterly Journal of Economics. Camerer and his team have worked with the original authors to design the replication experiments and have agreed in advance about what kinds of findings will constitute a confirmation and which will not. His team will also run a prediction market where knowledgeable economic experts can trade bets on the likelihood that various results are confirmed by the new data. The project will thereby not only measure whether these 18 experimental results can be replicated, but whether and to what extent the community of economists is able to reliably predict such replication when it is likely to happen and whether expert confidence serves as a good indicator of future replicability in economics.

This grant supports the continued operation and administration of InsideSchools.org, a public website that provides comprehensive information on New York City’s 1,700 public schools, including photos and videos of the school, student achievement statistics, course offerings, and reviews compiled by independent reviewers from on-site visits. Grant funds provide three years of core operational support as well as planned efforts to improve the site’s search capabilities and accessibility via smartphones and other mobile devices. In addition, the grant provides resources to help the site develop and implement plans for long-term financial sustainability.  

Funds from this grant support an initiative by the Business-Higher Education Forum (BHEF) to expand the number of NYC metro area institutions involved in educating undergraduates to become data scientists and data science–enabled professionals. Over the next four years, BHEF will convene and support the NYC Data Science Task Force of approximately 40 representatives from academic institutions, corporations, cultural and research organizations, and government agencies; convene two working groups, one aimed at mapping the skills, competencies, and knowledge needed for data scientists and one on developing a repository of undergraduate data science curricular resources; partner with NYC institutions to create data-science-focused courses, concentrations, and minors; work with industry partners to create high-quality internships and other student work experiences in data science and create guidelines and best practices for the creation of these experiences; and disseminate lessons learned to the broader educational community.  

This grant supports a project by UCLA Professor of Information Studies Christine Borgman to investigate the role of three key variables that influence the circulation of data in a given scientific community: diversity of disciplines, degree of centralization of data collection, and scale of data (i.e., “big” vs. “long-tail”). Through a set of research sites drawn from astronomy, ocean science, and biomedicine, and leveraging over a decade of data collected and coded from additional research sites, Borgman and her team will chart how these three attributes influence data practices. The resulting work will shed light on how the structure of scientific collaborations affects the willingness to share data, and help identify those areas of the scientific enterprise that may be more or less amenable to widespread data sharing. In addition to academic publications, Borgman’s work will produce implementable guidelines that could inform the design of future efforts by private and government funders interested in increasing data sharing in the sciences.

Recent work by Jim Herbsleb at Carnegie Mellon University found that volunteer contributions to open source software development projects increased in the aftermath of “community code engagement” (CCE) events like hackathons or summer coding projects. Yet little is known about how exactly CCEs lead to more contributions from volunteers, what makes for a good CCE, and what pitfalls to avoid. This grant funds efforts by Jim Herbsleb to continue his examination of how CCEs spur contributions to scientific software development and to compile a list of best practices for CCE design and implementation. Over the next three years, Herbsleb and his team will study successful and failed CCEs through participant observation, semistructured interviews, and quantitative analysis of software version histories to determine contribution patterns. He will then develop a set of best practices for CCE design and test these guidelines in a series of pilot projects.  Herbsleb and his team will then develop a CCE Toolkit that they will introduce to scientific software developers at a series of workshops attached to disciplinary meetings. The project promises to provide useful new information on how to spur engagement in community software development, an activity that is likely to become increasingly important as science moves further and further into the information age.

Bayesian statistical analysis is powerful, yet it is infrequently used in many scientific domains. Calculating Bayesian probability distributions is complicated, and available computer programs designed to do the job are slow and inefficient. As a result, a useful intellectual tool for the scientific analysis of data lies largely untapped. This grant supports development of Stan, a powerful, open source computing platform designed by Columbia University statistician Andrew Gelman that calculates Bayesian probabilities quickly and efficiently. Funds from this grant will support Gelman’s efforts to build out the capabilities of Stan, allowing it to seamlessly interact with other computing platforms like R, Python, and Julia that see wide use in the scientific community. Additional funds support development of Stan’s technical capabilities, allowing it to efficiently handle certain complex statistical models and community development and outreach through the organization of conferences and online users groups.