This grant funds a project lead by California Institute of Technology economist Colin Camerer to attempt to replicate the findings of 18 seminal papers in economics. Working with the original authors, Camerer has selected highly influential, highly cited papers that all deal with between-subject treatment effects that appeared between 2011 and 2014 in either the American Economic Review or the Quarterly Journal of Economics. Camerer and his team have worked with the original authors to design the replication experiments and have agreed in advance about what kinds of findings will constitute a confirmation and which will not. His team will also run a prediction market where knowledgeable economic experts can trade bets on the likelihood that various results are confirmed by the new data. The project will thereby not only measure whether these 18 experimental results can be replicated, but whether and to what extent the community of economists is able to reliably predict such replication when it is likely to happen and whether expert confidence serves as a good indicator of future replicability in economics.

This grant supports the continued operation and administration of InsideSchools.org, a public website that provides comprehensive information on New York City’s 1,700 public schools, including photos and videos of the school, student achievement statistics, course offerings, and reviews compiled by independent reviewers from on-site visits. Grant funds provide three years of core operational support as well as planned efforts to improve the site’s search capabilities and accessibility via smartphones and other mobile devices. In addition, the grant provides resources to help the site develop and implement plans for long-term financial sustainability.  

Funds from this grant support an initiative by the Business-Higher Education Forum (BHEF) to expand the number of NYC metro area institutions involved in educating undergraduates to become data scientists and data science–enabled professionals. Over the next four years, BHEF will convene and support the NYC Data Science Task Force of approximately 40 representatives from academic institutions, corporations, cultural and research organizations, and government agencies; convene two working groups, one aimed at mapping the skills, competencies, and knowledge needed for data scientists and one on developing a repository of undergraduate data science curricular resources; partner with NYC institutions to create data-science-focused courses, concentrations, and minors; work with industry partners to create high-quality internships and other student work experiences in data science and create guidelines and best practices for the creation of these experiences; and disseminate lessons learned to the broader educational community.  

This grant supports a project by UCLA Professor of Information Studies Christine Borgman to investigate the role of three key variables that influence the circulation of data in a given scientific community: diversity of disciplines, degree of centralization of data collection, and scale of data (i.e., “big” vs. “long-tail”). Through a set of research sites drawn from astronomy, ocean science, and biomedicine, and leveraging over a decade of data collected and coded from additional research sites, Borgman and her team will chart how these three attributes influence data practices. The resulting work will shed light on how the structure of scientific collaborations affects the willingness to share data, and help identify those areas of the scientific enterprise that may be more or less amenable to widespread data sharing. In addition to academic publications, Borgman’s work will produce implementable guidelines that could inform the design of future efforts by private and government funders interested in increasing data sharing in the sciences.

Recent work by Jim Herbsleb at Carnegie Mellon University found that volunteer contributions to open source software development projects increased in the aftermath of “community code engagement” (CCE) events like hackathons or summer coding projects. Yet little is known about how exactly CCEs lead to more contributions from volunteers, what makes for a good CCE, and what pitfalls to avoid. This grant funds efforts by Jim Herbsleb to continue his examination of how CCEs spur contributions to scientific software development and to compile a list of best practices for CCE design and implementation. Over the next three years, Herbsleb and his team will study successful and failed CCEs through participant observation, semistructured interviews, and quantitative analysis of software version histories to determine contribution patterns. He will then develop a set of best practices for CCE design and test these guidelines in a series of pilot projects.  Herbsleb and his team will then develop a CCE Toolkit that they will introduce to scientific software developers at a series of workshops attached to disciplinary meetings. The project promises to provide useful new information on how to spur engagement in community software development, an activity that is likely to become increasingly important as science moves further and further into the information age.

Bayesian statistical analysis is powerful, yet it is infrequently used in many scientific domains. Calculating Bayesian probability distributions is complicated, and available computer programs designed to do the job are slow and inefficient. As a result, a useful intellectual tool for the scientific analysis of data lies largely untapped. This grant supports development of Stan, a powerful, open source computing platform designed by Columbia University statistician Andrew Gelman that calculates Bayesian probabilities quickly and efficiently. Funds from this grant will support Gelman’s efforts to build out the capabilities of Stan, allowing it to seamlessly interact with other computing platforms like R, Python, and Julia that see wide use in the scientific community. Additional funds support development of Stan’s technical capabilities, allowing it to efficiently handle certain complex statistical models and community development and outreach through the organization of conferences and online users groups.

This grant funds three projects by the University of Chicago’s Macro-Financial Modeling (MFM) initiative. Led by University of Chicago economist and Nobel laureate Lars Peter Hansen and Andrew Lo of MIT, the MFM initiative is a group of distinguished economists, business professors, and other finance experts who have come together to meet the challenges of modeling the complex interactions between the real economy and modern financial institutions. The first supported project is a summer school for graduate students, which will bring young scholars from a variety of intellectual backgrounds to the University of Chicago to introduce them to macro-finanical modeling and to work on specific projects related to it. The second is an open call competition for new or crowd-sourced solutions to problems posed by the MFM initiative. The call will elicit the best thinking from outside the group, encourage innovative and creative approaches to established problems, and expand the reach of the initiative to those not yet involved in the program.  The third project is the development and construction of an online platform for comparing and archiving various macro-financial models. This platform will allow MFM scholars to compare, contrast, and evaluate different models and will spur integrative work that may lead to the combination or improvement of existing models.

The New York City Department of Health and Mental Hygiene (DOHMH) estimates that more than 1,000 restaurant-associated outbreaks of foodborne illness occur in the city each year. Outbreaks are usually reported by the victims themselves via telephone calls to 311 or the health department. Most victims don’t bother, however, and as a result the DOHMH detects only about 30 outbreaks each year. Since quickly detecting foodborne illness outbreaks is critical to implementing control measures in time to protect the public, better detection measures are needed. This grant funds a project by the Fund for the City of New York, in collaboration with the DOHMH and researchers at Columbia University to experiment with using Twitter and other social media to detect unreported instances of restaurant-related foodborne illness. The theory is that while people may be unlikely to report a foodborne illness to the health department, they are much more likely to tweet or post to Facebook about it. Real-time analysis of public data from Twitter and other social media sites may be able to reliably inform health department officials of outbreaks as they are happening. Over the next three years, the FCNY team will develop algorithmic methods for searching Twitter feeds, identifying tweets potentially relevant to foodborne illness outbreaks in NYC, and then evaluate the reliability of those algorithms in detecting actual outbreaks. Additional grant funds support efforts to increase voluntary reports of foodborne illness outbreaks by allowing NYC residents to report illness directly through Twitter. The project is experimental, but the prospective gains are large. Even a small increase in the ability to detect restaurant-related foodborne illness outbreaks would represent a significant improvement of current detection capabilities.